Applicant essays and recommendation letters in residency selection:How generative artificial intelligence is changing them

To the Editor

“How our writing reflects who we are,” as Buffon famously said: “Le style, c’est l’homme même.” We, Japanese, embrace a broader interpretation: “writing reveals the writer.”

As a previous Japanese obstetrics-gynecology professor, who wrote over 590 PubMed-indexed papers, I had confidence “if I look at one’s writing, I can discern one’s character and intellectuality,” which holds true for resident application essays: Alas! This was true 2 years ago. The emergence of generative artificial intelligence (GenAI, like ChatGPT) broke my beliefs and has changed the landscape surrounding writing in general. Here, GenAI and ChatGPT are used interchangeably.

Smith et al.’s study ¹ is highly significant. For brevity, I use the term application “essay” to represent personal statements + impactful experiences. They showed: a significant proportion of medical students did, or plan to, use GenAI to write essays for resident applications. Some knew their senior personnel used GenAI to make recommendation letters. Smith et al. were concerned about the future landscape of the resident selection, questioning the future role of essays and recommendation letters. I agree with their view. I add some points and suggest their future roles.

First, some studies showed that in both applicant essays^2,3 and recommendation letters, ⁴ reviewers couldn’t discern ChatGPT-generated from human-written ones. Interestingly, letters perceived as human-written (even if GenAI-generated) were more likely to receive higher scores. ⁴ As Smith et al. suggested, one may consider that “applicant unique journey” is “uniquely human” and thus will reflect one’s character. This is not always true, as demonstrated below.

Second, I demonstrated that, with appropriate input, ChatGPT-generated manuscripts with a “human touch”: personal anecdotes (experiences, beliefs, and proverbs alongside one’s life). ⁵ Initially, I believed: a human touch was less accessible to ChatGPT and this touch might characterize human writing. ⁶ However, with appropriate input, ChatGPT-generated Letters that were as appealing as those written by a veteran human author. ⁵

Third, as Smith et al. pointed out, the GenAI detector is far from complete and thus cannot be relied upon. Rather, the GenAI detector may erroneously classify human-written manuscripts as ChatGPT-generated, especially for nonnative writers. ⁷

Fourth, applicants whose native language is not English may struggle to fully utilize GenAI or may even misuse it. This nuance contradicts Smith et al.’s statement. Nonnatives, especially less-experienced ones, may be attracted to essays “beautifully” written by ChatGPT. Appropriately editing ChatGPT-generated manuscripts requires significant English ability. ⁸ Even after 46 years of writing, I still feel obliged to check the dictionary for ChatGPT-edited manuscripts. Words that require a dictionary should be avoided: veteran writers may understand this, but less-experienced ones may not.

Fifth, academic writing will become “monotonous,” losing “heterogeneity,” ⁹ a fundamental element of scientific progress. GenAI can be employed for generating application essays (applicant side) and recommendation letters (faculty side), and even for evaluating these pieces (department side). If a significant fraction of these three groups employs GenAI, GenAI style, rather than individual human-writing styles, may become the “norm.” Many may seek to become “cleverer users” of ChatGPT, losing individuality and heterogeneity in writing.

Students are already preparing to use GenAI in essay writing. ¹ GenAI can generate appealing manuscripts with a human touch, ⁵ and AI detectors are unreliable. ⁷ GenAI’s ubiquity and strong capabilities will make its use in writing essays unavoidable. Importantly, these developments can occur at the very start of a doctor’s career. This will influence not only resident selection but also medicine as a whole. Therefore, we must adhere to transparency.

Let’s assume that the selecting sides continue to use the traditional approach: (1) stipulating AI usage regulations in the application guidelines; (2) scrutinizing whether a piece is AI-generated or human-written; and (3) comprehensively evaluating the pieces, including their nuances.

To me, the solution is straightforward:

Essays and recommendation letters should be limited to “bullet points,” removing subjective factors (“how it’s written”) as much as possible.

AI regulations should not be specified in application calls; whether to use AI should be left to the applicants’ and recommenders’ discretion.

Reviewers should disregard whether the content is AI-generated or human-written.

By focusing solely on facts, reviewers are freed from the detective work of discerning whether a piece is “AI- or human-written.” This shift means applicants and recommenders may no longer need to rely on AI, as only objective “facts” will matter; writing them doesn’t require AI aid. Smith et al.’s study suggested this: students were less likely to use GenAI in writing curriculum vitae, a “fact.” Reviewers can evaluate the pieces purely for their factual content. I respect the ERAS system, and also the traditions and policies of individual institutions in their residency selection processes. I don’t consider my proposal to be the only solution; however, this approach to simplifying these pieces may offer valuable discussion points.

This may yield second-order benefits. As mentioned, AI detectors often misidentify human-written manuscripts as ChatGPT-generated, especially from nonnative speakers. ⁷ Allowing AI use would eliminate the need for such detectors, thus avoiding false accusations. With no regulations, applicants and recommenders face less burden and needn’t fear blame for using AI.

Smith et al. cautiously avoided concluding how to deal with essays and recommendation letters. I believe that, whether one likes it or not, there seems to be no choice but to diminish the weight of essays and recommendation letters in the residency application process.^10,11 Naturally, the weight of in-person interaction may increase, as Smith et al. suggested. A virtual face-to-face interview could be one option.

In the AI era, the chance to fully utilize my skill honed over decades has been nearly lost—my ability to recognize “writing reveals the writer.” As a retired professor, I reflect on my days as a member of my university’s residency selection committee. I feel nostalgic for the era when we valued both “what was written” and “how it was written.” It allowed us to glimpse the applicant’s personality and the recommender’s personal perspective. However, let’s consider that the COVID-19 pandemic reshaped medical practices, making virtual meetings a norm. Similarly, the rise of AI compels us to adapt. Times have changed.