Availability and Readability Level of Online Patient Education Materials Provided by Cleft Lip and Palate Teams

Abstract

Objectives

Evaluate the readability of online English and Spanish cleft lip and palate patient education materials.

Design

Review of free online materials

Setting

English and Spanish language online patient education materials on cleft lip and palate were collected from American Cleft Palate-Craniofacial Association (ACPA) approved teams.

Participants

American Cleft Palate-Craniofacial Association (ACPA) approved teams

Interventions

English materials were analyzed using the Flesch-Kincaid, SMOG, and Coleman-Liau readability calculators. Spanish materials were analyzed using the Fry Graph, Fernandez Huerta, and INFLESZ calculators. A one-way analysis of variance (ANOVA) was used to test for variability between the readability tools.

Outcomes

Readability levels were examined for both sets of materials.

Results

171 (90.5%) teams provided English language materials online, with an average readability score calculated as 10.5 ± 2.9 (10th-11th grade). A total of 44 (23.2%) teams listed Spanish language materials online, with average readability score of 7.9 ± 1.2 (8th grade). ANOVA demonstrated statistically significant variability between the readability assessment tools (P < .01).

Conclusion

Online cleft lip and palate patient education material provided by ACPA craniofacial teams were more available in English than in Spanish. Both sets of materials demonstrated readability levels above the recommended 6th-7th grade. Refining readability is associated with lowered healthcare costs and increased patient satisfaction.

Keywords

cleft lip cleft lip and palate cleft palate craniofacial surgery patient satisfaction

Introduction

The Centers for Disease Control and Prevention (CDC) defines health literacy as the “degree to which individuals have the ability to find, understand, and use information and services to inform health-related decisions and actions for themselves and others.”¹ Low health literacy is associated with increases in healthcare costs and hospitalization length, as well as reduced patient satisfaction.² Over one-third of American adults rely on the Internet for information regarding their own health or those of family members.³ This rise in online health information poses challenges to both providers and patients, as understanding, and applicability of this information to health care decision making is often a difficult task. The National Institutes of Health recommend that patient education materials should be written at a 6-7^th grade reading level to meet the reading level of the average United States (U.S.) adult.⁴

Treatment of cleft lip and palate is best provided through the work of multidisciplinary teams to maximize outcomes.⁵ Comprehensive cleft care involves professionals from primary care, plastic surgery, ENT, speech pathology, dentistry, radiology, audiology, among others,⁶ Important conversations regarding treatment options and decisions must take place from the start of diagnosis and continue for several years, even decades, through patient growth. Thus, both caregivers and the patient will be involved in the education and decision-making process regarding cleft care. To best provide information regarding treatment options, patient education materials (PEMs) are developed to aid conversations and promote relationships between patients and providers, as well as enhance patient and caregiver understanding. With 13% of the U.S. population being Spanish speaking,⁷ it is also important to provide PEMs that meet the needs of a diverse patient population with a diagnosis of cleft lip and/or palate.

The American Cleft Palate-Craniofacial Association (ACPA) seeks to “empower and support cleft and craniofacial healthcare professionals and the patients they serve through research, education, and advocacy.”⁸ ACPA provides an online database of names and contact information of approved teams in the United States. ACPA-approved teams undergo a review process that evaluates six key components: team composition, team management and responsibilities, cultural competence, patient and family/caregiver communication, psychological and social services, as well as outcomes assessment.⁹ ACPA also provides information on patient education and cleft care for both health care professionals and patients. Given the multitude of online information regarding treatment options for cleft lip and/or palate, the availability of evidenced-based PEMs authorized by ACPA and ACPA-approved teams is an important part of the decision-making process for patients and caregivers.

Previous studies on both web-based and printed English-language PEMs related to cleft lip and palate care reveal that most exceed the recommended reading level.^10–16 However, there have been fewer studies exploring the availability and readability of multilingual patient education materials on cleft lip and palate. In prior studies on the availability of online cleft lip and palate PEMs provided by ACPA-approved teams, it was found that less than half of the teams provided were in languages other than English.^17,18 To the authors’ knowledge, there have been no prior studies specifically analyzing the readability of Spanish PEMs within cleft lip and palate care. Readability of PEMs can be assessed using several different readability measures. Each of the measures used in this study incorporates metrics such as the number of words per sentence, the number of syllables per word, the number of complex words, and sentence length to calculate a reading level. Given the importance of health literacy in medical decision making, this study aims to evaluate and compare online cleft lip and palate PEMs written in both English and Spanish by ACPA-approved teams.

Methods

Search Strategy

This study utilized online publicly available resources, qualifying for IRB exemption. Cleft lip and/or palate teams in the United States were identified through the ACPA's list of approved teams. The ACPA's website includes team names, locations, contact information, designated team leaders, and a link to the team's website. The listed websites were accessed and evaluated for presence of free online English and Spanish PEMs on cleft lip and palate. If the ACPA did not provide a link or if the links were nonfunctional, a Web-based search was used to locate the team's materials. The search queries were “cleft lip and palate” in addition to the team's name for English resources; for Spanish resources, “labio leporino y paladar hendido” and “fisura labiopalantina” in addition to the team's name were searched. These search terms were based on the wording used in Spanish-language materials directly available on ACPA team websites. Research on Google click-through rates have shown that the top three Google search results receive approximately 75% of all clicks.¹⁹ To emulate this, the first resource identified from the team's website using this search strategy was used for readability analysis. The PEMs identified were not limited to a specific age group or topic within cleft care. If the first resource identified did not meet criteria for readability analysis (fewer than 100 words), the next result from the ACPA team's website or a Google result was used. The search browser utilized in the study was cleared to limit potential result bias, including search engine history, cookies, location services, and user account information.

Evaluation of Readability

A minimum of 100 words is needed to perform a readability analysis based on criteria from the readability calculators used in this study. The online materials were copied in plain-text format and open-source software cited in numerous readability studies was utilized to calculate readability (https://readabilityformulas.com).^20–23 The English materials collected were analyzed using three different validated readability scales: Flesch-Kincaid Grade Level, Simplified Measure of Gobbledygook (SMOG) Readability Formula, and Coleman-Liau Index. The Flesch-Kincaid Grade Level considers the number of words per sentence and the number of syllables per word.²⁴ The SMOG (Simplified Measure of Gobbledygook) Readability Formula assesses the number of polysyllabic words (ie, three or more syllables) in a total of 30 randomly selected sentences from the text.²⁵ The Coleman-Liau Index assesses the average number of letters per 100 words and the average number of sentences per 100 words.²⁶

The Spanish materials collected were analyzed using three different validated readability scales. These scales included the Fry Graph Score, which assesses the number of sentences and syllables in three, random 100-word passages from the selected text²⁷; Fernandez Huerta Score, which is a Spanish adaptation of the Flesch Reading Ease score for analyzing texts in English. The Fernandez Huerta Score calculation is based on syllable count and sentence length in the first 100 words of a sample text²⁸; and Índice Flesch-Szigriszt (INFLESZ) score, which is adapted from the Flesch-Szigriszt for Spanish texts which incorporates the number of syllables, words, and sentences in a text to calculate a reading level.²⁹ Formulas for each of the Spanish and English readability tools are shown in Table 1.

Table 1.

Readability Calculators Utilized for English and Spanish Language Education Materials.

Language and Index	Formula
English
Flesch-Kincaid Index	G = (0.39 x average number of words/sentence) + (11.8 x average number of syllables/word) – 15.59
Simple Measure of Gobbledygook (SMOG)	G = 1.0430 x X* + 3.1291
Coleman-Liau Index	G = 0.0588(average number of letters/100 words) - 0.296(average number of sentences per 100 words) - 15.8
Spanish
Fry-Graph Score	G = Select three samples of 100-word passages. Take the average number of words per 100 words (x-axis) and average number of syllables per 100 words − 67 (y-axis) plotted onto a Standard Fry Graph.
Fernandez Huerta Score	G = 206.84 − (0.6 × total number of syllables) − 1.02 × (total number of words)
INFLESZ Score	G = 206.835–62.3 syllables/words – words/phrases

X*, words of 3 or more syllables; G, grade level. INFLESZ, Índice Flesch-Szigriszt Score.

Statistical Analysis

Statistical analysis was performed using JMP 15.1 (SAS Institute, Inc, SAS Campus Drive, Cary, NC) and Microsoft Excel Version 16.61 (Microsoft Corporation One Microsoft Way Redmond, WA). The Fernandez Huerta and INFLESZ Spanish language readability tools were converted from a raw score to a reading level score to facilitate statistical analysis.^28–30 Mean readability was calculated for each of the English and Spanish readability assessment tools, along with standard deviation. P value was set at .05. An unpaired, two-tailed t test was used to compare the mean for the independent samples, English PEMs and Spanish PEMs. P value was set at .05. One-way Analysis of Variance (ANOVA) was used to assess variability among the readability calculators for both English and Spanish. The scores were given as means ± standard deviation. The average readability of English PEMs by teams providing both English and Spanish materials was then compared with the average readability of English PEMs by teams providing English language materials alone. P value was set at .05.

Results

A total of 189 ACPA-approved teams were identified. A summary of the number of teams providing English and Spanish PEMs is provided in Table 2. Several topics were addressed in the PEMs, including general information (eg, diagnosis, causes), surgical treatment, and additional specialty specifics (eg, dentistry, speech therapy). An overview of the topics discussed can be found in Table 3. Of these 189 ACPA-approved teams, 171 (90.5%) provided online English language materials. The mean readability scores of English-language resources include a Flesch-Kincaid of 10.9 ± 2.8 (equivalent to a 10^th-11^th grade reading level; range: 6-26), SMOG Readability of 9.8 ± 2.8 (equivalent to a 9^th-10^th grade reading level; range: 5.5-21.4), and Coleman-Liau Index of 10.8 ± 3.7 (equivalent to a 10^th-11^th grade reading level; range: 4-19). The average readability of cleft lip and palate English materials by the three assessment tools was 10.5 ± 2.9 (equivalent to a 10^th-11^th grade reading level). These results are displayed in Figure 1.

Figure 1.

Bar graph showing the mean reading grade level for English language patient education materials from ACPA-approved team websites. Reading level was assessed with three validated metrics: Coleman-Liau, SMOG, and Flesch-Kincaid. Mean readability was calculated at: Coleman-Liau Index of 10.8 (± 3.7), SMOG Readability of 9.8 (± 2.8), and Flesch-Kincaid of 10.9 (± 2.8). The average readability of cleft lip and palate English materials by the three assessment tools was 10.5 ± 2.9. SMOG, Simplified Measure of Gobbledygook.

Table 2.

Online English and Spanish Patient Education Materials Provided by ACPA Cleft Teams.

Total Number of ACPA Sites Identified, N	Number of Sites that Provide English Materials, N (%)	Number of Sites that Provided both English and Spanish Materials, N (%)	Number of Sites that Provided Spanish Materials Only, N (%)
189	171 (90.5%)	44 (23.2%)	0 (%)

Table 3.

Distribution of Topics Covered in English and Spanish Patient Education Materials.

Topics Discussed	English N (%)	Spanish N (%)
Only General Information^a	102 (59.6%)	25 (56.8%)
Only Surgical Treatment	5 (2.9%)	3 (6.8%)
Only Additional Specialty Specifics	1 (0.6%)	1 (2.3%)
General Information + Surgical Treatment + Additional Specialty Specifics	24 (14.0%)	5 (11.4%)
General Information + Surgical Treatment without Additional Specialty Specifics	39 (22.8%)	10 (22.7%)
General Information + Additional Specialty Specifics without Surgical Treatment	0 (0%)	0 (0%)
Total	171	44

Only General Information refers to resources covering diagnosis, risk factors, causes, treatment with basic surgical information, and general interdisciplinary needs

Of the 189 ACPA-approved teams, a total of 44 (23.2%) teams provided online Spanish language materials. The mean readability scores of Spanish-language resources include a Fry-Graph score of 6.1 ± 1.2 (equivalent to a 6^th-7^th grade reading grade level; range: 4-9), a Fernandez Huerta Score of 65.4 ± 11.4 (equivalent to an 8^th-9^th grade reading level; range: 6.5-13.2), and an INFLESZ Score of 60.8 ± 7 (equivalent to a 6^th-8^th grade reading level; range: 5.5-12). The average readability of cleft lip and palate Spanish materials by the three assessment tools was 7.9 ± 1.2 (equivalent to an 8^th grade reading level). These results are displayed in Figure 2.

Figure 2.

Bar graph demonstrating mean Reading grade level for Spanish language patient education materials from ACPA-approved team websites. Reading level was assessed with three validated metrics: Fry-Graph, Fernandez Huerta, and INFLESZ. Mean readability was calculated at: Fry-Graph score of 6.1 (± 1.2), a Fernandez Huerta Score of 65.4 (± 11.4), and an INFLESZ Score of 60.8 (± 7). The average readability of cleft lip and palate Spanish materials by the three assessment tools was 7.9 (± 1.2). INFLESZ, Índice Flesch-Szigriszt Score.

The present study showed a statistically significant difference between the mean readability score of English PEMs and Spanish PEMs, with English PEMs having a higher readability score (10.5 ± 2.9 vs. 7.9 ± 1.2, respectively). One-way ANOVA demonstrated statistically significant variance among the English readability tools (F = 7.6, df = 2,168, P < .01) and Spanish readability tools (F = 64.6, df = 2,41, P < .01).

Teams providing PEMs in both languages had a mean English readability score of 9.1 (SD = 1.6, 9th-10th grade reading level). Teams only providing English materials had a mean readability score of 11.0 (SD = 2.9, 11th grade reading level). An unpaired, two-tailed t test showed statistical significance in the average readability between the two groups (P < .001).

Discussion

Low health literacy is associated with billions of dollars annually in increased hospitalization costs, as well as poor surgical outcomes and lowered patient satisfaction.^2,31 With more than one-third of Americans relying on the Internet as a source of health information,³ it is becoming increasingly important to assess the quality of the information provided online and its ability to convey information to the general population. The aim of this study was to identify and compare availability and readability of Spanish and English patient education materials on cleft lip and palate provided online by ACPA-approved teams.

More than 40 readability formulas exist, all of which apply varying algorithms to count syllables, words, and phrases in a given text.³² Given the variability in readability calculations from different readability formulas, at this time, there is not a general consensus on which readability formula should be used to assess PEMs. Thus, it is recommended to use several formulas to increase the validity of the results.³³ In the assessment of healthcare information, teams should use multiple readability formulas and the same readability calculators to maintain consistency among their studies. The six formulas utilized in this study have been used extensively in medical literature and have shown to have high intercorrelations across health information materials.³⁴ An additional consideration when selecting a readability tool is that language structure varies between non-English languages, and validated English language tools may not be appropriate for calculating readability in other languages. For example, many readability tools assign a certain numerical value to the number of syllables in a word, with a greater number of syllables corresponding with increased complexity. In English, “medicine” has three syllables, while in Spanish, “medicina” (the same word) has four syllables. Utilizing traditional English readability tools would erroneously assign a higher score to the word written in Spanish. Thus, when conducting readability analysis on non-English languages, it is of utmost importance to utilize tools which have been appropriately validated.^29,30,35

Non-English-speaking patients face several healthcare challenges. Prior studies have found that Latino children often receive fewer preventive services and are more likely to lack health insurance.³⁶ Additionally, these studies note that effectiveness of treatment may decline when patients have poor understanding regarding peri-treatment instructions.¹⁶ Particularly amongst surgical patients, proper communication is vital to understanding perioperative instructions. Prior research by Hernandez et al. surveyed Spanish speaking adults and found that the majority preferred an oral explanation of treatment options by their physician; however, 25% also requested written materials.³⁶ This study found that while the majority of teams provided materials written in English, less than one-fourth provided written Spanish language materials. English and Spanish PEMs did have a similar distribution of topics discussed. The majority of PEMs covered general information (eg, diagnosis, risk factors, causes) followed by PEMs that addressed general information and surgical treatment without additional specialty specifics. Because patients with cleft lip and/or palate and their caregivers will interact with a multitude of specialties throughout their lives, incorporating information that discusses the role of other specialties, such as speech therapists, dentists, and audiologists, into PEMs is essential. ACPA-approved teams have reported a rise in Spanish-speaking patients and thus, there is a need for accessible, readable patient education materials to meet the needs of this diverse population.³⁷

Computer-generated translation offers a potential cost-effective way to increase the accessibility Spanish PEMs and can easily be implemented into websites. However, these translation options are known to have inaccuracies and are generally inferior to translations by native speakers.^38,39 Translating information from English to Spanish may lead to omitting important information, replacing simpler terms with more technical and complex terms, or alter the meaning of the original text.⁴⁰ In a study that used Google Translate to translate emergency department discharge instructions from English to Spanish, 92% of sentences were accurately translated; however, 28% of the inaccuracies identified had the potential to cause significant harm.³⁹ These results show that while computer-generated translations can be useful, the use of these resources must be exercised with caution.

Of the 189 ACPA teams, only 23.2% of ACPA teams provided online PEMs in Spanish. Many states with large Spanish-speaking populations (eg, California, Texas, Florida, New York, Arizona) provided materials in both languages. California, having the largest proportion (28.3%) of Spanish speakers in the nation,⁴¹ had the greatest number of ACPA teams offering Spanish language materials (8/44 (18%)) of all states. However, several states with a large proportion of Spanish-speakers did not provide materials in both languages, indicating a continued need for PEMs that are accessible and understandable for diverse populations. Interestingly, PEMs from ACPA teams that provided both English and Spanish PEMs had lower readability scores compared to PEMs from teams that only provided PEMs in English. This finding may indicate increased emphasis on readability by teams providing diverse PEMs to target the recommended reading level.

Prior studies have explained several techniques that can be used to increase the readability of PEMs and thus improve patient understanding and contribute to better outcomes. These include: avoiding medical jargon, decreasing the use of complex words (ie, words with more than 3 syllables), limiting sentence length to 10-15 words, and using bulleted or numerical lists when possible.^37,42 Other methods to maximize understanding includes the use of headings and illustrations, use of increased font size to emphasize key points or headings, and by utilizing examples.⁴³ Supplemental Figure 1 displays a potential flow chart that can be used to write and evaluate PEMs that meets the recommended 6-7th grade reading level.^44–51 Table 4 displays potential word substitutions that can be used to improve readability of both English and Spanish language education materials.

Table 4.

Examples of English and Spanish Language Word Substitutions to Improve Readability.

Difficult Word	Suggested Word	Difficult Word	Suggested Word
Additional	More	Acumulación	Colección
Asymmetric	Uneven	Adecuado/a	Correcto/a
Component	Part	Anomalía	Defecto
Comprehensive	Complete, broad	Aprovechar	Utilizar
Development	Growth	Aproximadamente	Más o menos, alrededor de
Displacement	Move, shift	Congénito/a	Al nacer, de nacimiento
Etiology	Cause	Considerablemente	Mucho/a
Evaluate	Test, check	Evaluación	Exámen
Expertise	Skill	Exclusivamente	Solo, solamente
Extremely	Very	Hemorragia	Sangría
Frequently	Often	Malformación	Estructura anormal
Gestation	Pregnancy	Multidisciplinario/a	Colaborativo/a
Orofacial	Mouth and face	Ortognatica	Cirugía de la mandíbula
Requirement	Need	Proporcionar	Proveer
Recommend	Suggest	Queiloplastia	Cirugía de los labios
Sufficient	Enough	Recuperar	Mejorar
Suture	Stitch	Suficiente	Bastante

This study had several limitations. While written patient education materials are important resources for patients and families receiving information on cleft care, oral communication, pictures, videos, and other forms of healthcare information are all important contributors to perioperative decision making and guidance. Thus, the overall understanding a patient may have when reviewing any of these ACPA team materials may not fully be reflected in the calculated readability score. Additionally, this study was limited to materials written in English and Spanish. While cleft lip with or without cleft palate can occur in any race, there are higher rates amongst Asian, Hispanic, and Native American patients⁵²; thus providing patient education materials in several languages is important to meet the diversity of patients requiring cleft care. The readability calculators used in this study can analyze a maximum of 3000 words from a given text; thus, texts exceeding 3000 words may not be fully assessed, potentially leading to inaccurate readability calculations. Finally, the researchers did not limit the analysis of resources to a specific topic or age group. Variations in the target age groups and the topics covered in the resources may have contributed to discrepancies in readability among materials.

Future studies may analyze the availability of PEMs written in languages beyond English and Spanish, expand readability analysis to patient education materials on adult care following cleft lip and/or palate surgery (eg, orthognathic surgery, lip/nasal revision, etc), and assess patient preference for educational resources on cleft lip with or without cleft palate (eg, oral communication, written materials, phone application, video, or other forms of communication). Additionally, it would be interesting to compare the readability of PEMs from different specialties within the cleft care team (ie, speech pathology, dentistry, audiology). One may assume that readability and comprehension may vary among Latino subgroups (eg, Mexican American, South-Americans, Puerto-Ricans). There is scant literature available on Spanish language dialect variation in Spanish dialects in general, let alone in medical jargon and terminology. Further research may focus on the readability of Spanish language patient educational materials among different Latino subgroups.

Conclusions

The availability of online patient education materials on cleft lip and palate was much greater for materials written in English than in Spanish, as seen on the websites by ACPA-approved teams. Average readability of English PEMs by sites providing materials in both English and Spanish was more aligned with recommended reading levels than that of sites providing English materials alone; this suggests that the former may have an increased emphasis on readability. In an area of medicine in which proper communication is essential for maximizing outcomes, there will be an ongoing need for PEMs that can meet the diverse needs of the cleft lip and palate population.

Supplemental Material

sj-docx-1-cpc-10.1177_10556656231213170 - Supplemental material for Availability and Readability Level of Online Patient Education Materials Provided by Cleft Lip and Palate Teams

Supplemental material, sj-docx-1-cpc-10.1177_10556656231213170 for Availability and Readability Level of Online Patient Education Materials Provided by Cleft Lip and Palate Teams by Lauren E. Powell, Erica M. Bien, Jade M. Cohen and Ruth J. Barta in The Cleft Palate Craniofacial Journal

Footnotes

Acknowledgements

The authors would like to thank Todd Warner for his assistance with the statistical design and analysis of this study.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Erica M. Bien

Supplemental Material

Supplemental material for this article is available online.

References

What is health literacy? Centers for Disease Control and Prevention; March 31, 2023. Accessed June 13, 2023. https://www.cdc.gov/healthliteracy/learn/index.html

Ismail

Burns

Rezak

, et al. An evaluation of health literacy in plastic surgery patients. Plast Reconstr Surg. 2015;136(4S-1):55-59.

More than half of American households used the internet for health-related activities in 2019, NTIA data show. National Telecommunications and Information Administration; 2020, Accessed July 8, 2023. https://ntia.gov/blog/2020/more-half-american-households-used-internet-health-related-activities-2019-ntia-data-show

Hutchinson

Baird

Garg

. Examining the Reading level of internet medical information for common internal medicine diagnoses. Brief Observation. 2016 June;129(6):637-639.

Preidl

RHM

Kesting

Rau

. Perioperative management in patients with cleft lip and palate. Journal of Craniofacial Surgery. 2020;31(1):95-101.

Kasten

Schmidt

Zickler

, et al. Team care of the patient with cleft lip and palate. Curr Probl Pediatr Adolesc Health Care. 2008;38(5):138-158.

Bureau

USC

. Facts for features: Hispanic Heritage month 2017. Census.gov. Published 2021. Accessed June 11, 2022. https://www.census.gov/newsroom/facts-for-features/2017/hispanic-heritage.html

Vision & Mission Statement. American Cleft Palate-Craniofacial Association. 2021. Accessed May 22, 2022. https://acpa-cpf.org/mission-vision-values/

Standards for approval of cleft palate and craniofacial teams. American Cleft Palate-Craniofacial Association; February 2019. Accessed May 22, 2022. https://acpa-cpf.org/wp-content/uploads/2019/04/Standards-2019-Update.pdf

10.

Alfonso

DeMitchell-Rodriguez

Ramly

, et al. Assessment of American cleft palate-craniofacial association-approved Teams’ websites for patient-oriented content and readability. Cleft Palate Craniofac J. 2019;56(9):1213-1219.

11.

Wasserburg

Sayegh

Sanati-Mehrizy

, et al.

Cleft care readability: Can patients access helpful online resources?

Cleft Palate Craniofac J. 2021;58(10):1287-1293.

12.

Lavin

Fahy

Abbas

, et al. Readability of online patient information relating to cleft palate surgery. Cleft Palate Craniofac J. 2022 Mar;59(3):330-335.

13.

Coco

Colina

Atcherson

, et al. Readability level of Spanish-language patient-reported outcome measures in audiology and otolaryngology. Am J Audiol. 2017;26(3):309-317.

14.

Powell

Cisu

Klausner

AP.

Bladder cancer health literacy: Assessing readability of online patient education materials. Bl Cancer. 2021;7(1):91-98.

15.

Washington

Cepeda

Jr Moffitt

Groff

Greives

Nguyen

. Is the message clear? Evaluation of readability levels for cleft lip, cleft palate, and craniofacial websites. Cleft Palate Craniofac J. 2023;60(12):1619-1624 .

16.

De Felippe

Kar

. Readability of information related to the parenting of a child with a cleft. Interact J Med Res. 2015;4(3):e14.

17.

Sunkara

Powell

Quick

Landis

. A comprehensive assessment of cleft lip and palate websites and patient education information. Cleft Palate Craniofac J. 2023;0(0):1-8.

18.

Edwards

Bonilla

. The use of Spanish language educational materials by American cleft palate-craniofacial association teams. Cleft Palate Craniofac J. 2004;41(6):655-660.

19.

Shelton

. The Value of Search Results Rankings. Forbes. October 30, 2017. Accessed July 8, 2023. https://www.forbes.com/sites/forbesagencycouncil/2017/10/30/the-value-of-search-results-rankings/?sh=1185b9ec44d3

20.

Gardner

. Health literacy and patient safety events. Pa Patient Saf Advis. 2016;13(2):58-65.

21.

Wiener

Wiener Pla

. Evaluation of educational material for tobacco prevention and cessation used in West Virginia university dental programs. J Dent Hyg. 2011;85(3):204-210.

22.

Readability tools. Transition TN. 2023. Accessed July 8, 2023. https://transitiontn.org/downloadable/readability-tools-2/

23.

Fuentes-Tirado

Vélez-Torres

Colón-Santaella-Colon

Mulero-Portela

. Calidad de la información sobre ejercicios en las páginas de Internet dirigida a sobrevivientes de cáncer de seno. Rev Iberoam Fisioter Kinesiol. 2011;14(2):70-76.

24.

Flesch

. A new readability yardstick. Journal of Applied Psychology. 1948;32(3):221-233.

25.

McLaughlin

. SMOG Grading—a new readability formula. J Reading. 1969;12(8):639-646.

26.

Coleman

Liau

. A computer readability formula designed for machine scoring. Journal of Applied Psychology. 1975;60(2):283-284.

27.

Gillian

Peña

Mountain

. The fry graph applied to Spanish readability. Read Teach. 1980;33(4):426-430.

28.

Huerta

. Medidas sencillas de lecturabilidad. Consigna. 1959;214:29-32.

29.

Barrio-Cantalejo

Simón-Lorda

Melguizo

, et al. Validación de la Escala INFLESZ para evaluar la legibilidad de los textos dirigidos a pacientes. An Sist Sanit Navar. 2008;31(2):135-152.

30.

Gorrepati

Smith

. Contrasting readability and availability of Spanish language with English language patient education materials. Pediatr Dermatol. 2021;28(S2):142-143.

31.

De Oliveira

Jr McCarthy

Wolf

, et al. The impact of health literacy in the care of surgical patients: A qualitative systematic review. BMC Surg. 2015;15(1):86.

32.

Wilson

Williams

. Assessing the readability of skin care and pressure ulcer patient education materials. JWOCN. 2003;30(4):224-230.

33.

Badarudeen

Sabharwal

. Assessing readability of patient education materials: Current role in orthopaedics. Clin Orthop Relat Res. 2010;468(10):2572-2580.

34.

Ley

Florio

. The use of readability formulas in health care. Psychol Health Med. 1996;1(1):7-28.

35.

Castillo-Ortiz

, Valvidia-Nuno J, Ramirez-Gomez A, Garagarza-Mariscal H, Gallegos-Rios C, Flores-Hernandez G, Hernandez-Sanchez L, Brambila-Barba V, Castaneda-Sanchez JJ, Barajas-Ochoa Z, et al. Readability, relevance and quality of the information in Spanish on the web for patients with rheumatoid arthritis. Reumatología Clínica. 2017;13(3):139-144.

36.

Hernandez

Cruz

Robinson

. Spanish-speaking patient health education preferences. Arch Dermatol. 2011;147(2):242-244.

37.

Wernick

, Hale P, Anticich N, Busch S, Merriman L, King B, Pegg T. A randomised crossover trial of minimising medical terminology in secondary care correspondence in patients with chronic health conditions: Impact on understanding and patient reported outcomes. Intern Med J. 2016;46(5):596-601.

38.

Dew

Turner

Choi

Bosold

Kirchhoff

. Development of machine translation technology for assisting health communication: A systematic review. J Biomed Inform. 2018;85(C):56-67.

39.

Khoong

Steinbrook

Brown

Fernandez

. Assessing the use of google translate for Spanish and Chinese translations of emergency department discharge instructions. JAMA Intern Med. 2019;179(4):580-582.

40.

Brelsford

Ruiz

Beskow

. Developing informed consent materials for non-English-speaking participants: An analysis of four professional firm translations from English to Spanish. Clin Trials. 2018;15(6):557-566.

41.

Explore Census Data. data.census.gov. Accessed October 16, 2023. https://data.census.gov/profile/California?g=040XX00US06

42.

Sheppard

, Hyde Z, Florence MN, McGwin G, Kirchner JS, Ponce BA. Improving the readability of online foot and ankle patient education materials. Foot Ankle Int. 2014;35(12):1282-1286.

43.

Horner

Surratt

Juliusson

. Improving readability of patient education materials. J Community Health Nurs. 2000;17(1):15-23.

44.

Summers

. Making the web friendlier for lower-literacy users. Intercom (Des Moines). 2004;51(6):19-21.

45.

U.S. Department of Health and Human Services & U.S. General Services Administration. Home | usability.gov. Research-Based Web Design & Usability Guidelines. 2006. Accessed May 31, 2022. https://www.usability.gov/sites/default/files/documents/guidelines_book.pdf

46.

Health Literacy Studies: Developing materials. Harvard T. H. Chan School of Public Health. 2015. Accessed May 31, 2022. https://www.hsph.harvard.edu/healthliteracy/developing-materials/

47.

Hou

. Health literacy online: A guide to writing and designing easy-to-use health websites. Health Promot Pract. 2012;13(5):577-580.

48.

U.S. Centers for Medicare & Medicaid Services. Toolkit for making written material clear and effective. CMS. December 1, 2021. Accessed May 31, 2022. https://www.cms.gov/outreach-and-education/outreach/writtenmaterialstoolkit

49.

The Patient Education Materials Assessment Tool (PEMAT) and user's guide. AHRQ. October 2013. Accessed May 31, 2022. https://www.ahrq.gov/health-literacy/patient-education/pemat.html

50.

The CDC Clear Communication index. Centers for Disease Control and Prevention. 2021. Accessed May 31, 2022. https://www.cdc.gov/ccindex/index.html

51.

Doak

Root

Doak

. Teaching patients with low literacy skills. J. B. Lippincott; 1996.

52.

About cleft lip and palate. Nationwide Children's Hospital. Accessed May 22, 2022. https://www.nationwidechildrens.org/specialties/cleft-lip-and-palate-center/about-cleft-lip-and-palate

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.00 MB

0.05 MB