Abstract
Background and Objectives:
Short video platforms have become one of the most common methods for disseminating medical information on social media. We analysed gastrointestinal (GI)-related content on TikTok, focusing on the creators’ background, patterns of content utilisation and overall content quality and understandability, using validated metrics to quantify the degree of misinformation.
Methods:
Twelve hashtags related to common GI conditions on TikTok and the 20 most engaging posts associated with each hashtag were selected for review. The ‘most engaging’ status was determined by the application’s internal algorithm. Six GI board-certified physicians evaluated the videos using two validated instruments: DISCERN quality criteria and Patient Education Materials Assessment Tool (PEMAT-A/V) survey.
Results:
In total, 253 videos were reviewed with 18.6 billion likes, 191,000 comments, 935,000 saves and 927,000 shares. IBS hashtag had the greatest number of views (2.2 billion), while videos on haemorrhoids had the most likes (4.3 million), comments (59,000) and shares (288,000). Most videos 177/253 (70%) were created by non-healthcare workers. The median video quality was 2/5 on DISCERN Likert-type scale with 23/75 DISCERN score. Of 186 videos, 114 (61.3%) were rated ‘very poor’; 58/186 (31.2%) ‘poor’, 13/186 (7%) ‘fair’ and only one video (0.5%) was rated ‘excellent’ on DISCERN grading system. The median PEMAT-A/V scores were 7/12 (73.9%) for understandability and 1/3 (33.3%) for actionability.
Conclusion:
Our study highlights significant public interest in common GI disorders. However, most posts lacked substantive information and did not have verifiable sources.
Introduction
Over the past decade, social media have played a significant role in shaping public perspectives on healthcare and health conditions. The modern iteration of the Internet – characterised by instantaneous access to innumerable opinions, sources and information – now provides people with the tools to explore their health concerns without structured professional guidance. A 2014 Pew Research report discovered that nearly three-quarters (72%) of US adults search for health information online, with a majority attempting to find specific diagnoses and treatments (Clarke et al., 2016; Fox, 2014).
Among the plethora of digital options available to patients and other people seeking information about health information is social media, which has seen a substantial rise in usage as a primary source of health information in the public domain following the COVID-19 pandemic (Zhong et al., 2020). Social media platforms with a short video format, such as TikTok, are especially popular among young adults owing to their accessibility and easily comprehensible content. Despite the benefits of expanding and increasingly accessible public medical knowledge, modern physicians must now address patients’ overreliance on these platforms as a primary source of medical information (Anderer, 2024). Widespread engagement with health-related content raises concerns about the quality of health information available and disseminated on these social media platforms. In the early 2000s, the term infodemic was coined after the 2002 SARS outbreak to refer to a rapid increase in information dissemination (both accurate and inaccurate) surrounding a particular topic. The term later rose to prominence during the COVID-19 pandemic as massive amounts of false and invalidated information were disseminated, especially during the early phases of the pandemic when information was scarce. Social media platforms such as TikTok, which further contribute to the rapid dissemination of potentially inaccurate health information, have paved the way for infodemics to now involve everyday medical conditions. To address this growing concern, the World Health Organization (WHO) has published a manual with recommendations on how to perform evidence-based analysis and navigate the complexities of a growing health information pool (WHO, 2023).
Prior studies have analysed misinformation in social media content within the disciplines of dermatology, gynaecology, plastic surgery and urology (Adhikari et al., 2016; Lenczowski and Dahiya, 2018; Rivera et al., 2022; Tam et al., 2022; Xu et al., 2021; Zekster et al., 2019). These studies all found substantial interest from the public in the subject matter investigated, but also that most of the posts lacked substantive health information that could be useful to patients. Specifically, one study evaluating the quality of information on TikTok regarding prostate cancer found that only 4% of the posts were made by physicians, and 98.2% of the videos were graded moderate to poor in quality, accounting for 134,752 views. In another study focusing on information about plastic surgery on TikTok, 47.6% of the evaluated posts were created by non-physicians, while 60.2% were identified as self-promotional rather than purely educational. In addition, the authors found that among the videos with objective information, 47% contained a significant amount of misinformation (Xu et al., 2021). Another urological study evaluated the quantity of misinformation on YouTube and TikTok related to urinary tract infections (UTIs). The authors found that videos on TikTok with fewer likes contained less misinformation based on a 5-point Likert-type scale compared to videos with a greater number of likes. The authors found that the more popular videos on TikTok contained a much fewer number of educationally oriented videos (42% vs 94%) and had a lower median misinformation score (3.33 vs 5) than the less popular videos (Tam et al., 2022).
Data on social media content related to the discipline of gastroenterology remain limited. There is evidence of growing interest in gastrointestinal disorders among young populations, with TikTok hashtags such as ‘#guttok’ or ‘#guthealth’ receiving 728 million and 3 billion views, respectively, as of January 2023 (TODAY, 2024). With the potential for social media to both positively and negatively affect patients’ perceptions of healthcare, while simultaneously serving as a valuable tool through which patients can learn more about health conditions, it is important to identify what constitutes good quality media (Patrick et al., 2022). We sought to investigate gastroenterology-related content on the short video social media platform TikTok, focusing on the background of the content creators behind the posts and patterns of content utilisation. We used validated metrics to quantify the actionability, understandability and quality of these video posts.
Methods
Data collection
We evaluated TikTok videos related to common gastrointestinal disorders between March and April 2023. The social media platform was accessed both via desktop and mobile iOS applications. The authors engaged in initial data collection over a month period, using 12 different ‘hashtags’ as search items. The hashtags searched were chosen by the authors ahead of time and included common gastrointestinal disorders. We then selected and analysed the 20 ‘most engaging’ posts associated with each hashtag. The ‘most engaging’ status is determined by the application’s internal algorithm based on a composite number of ‘likes, comments, views, and shares’. Posts of this status are featured on the platform’s ‘top’ page.
Video classification
The data extracted for each post included the engagement metadata (likes, comments, views and shares). Other demographic data were extracted, including the content creator type (physician individual, non-physician healthcare worker, non-healthcare worker and corporation) and the purpose of the post (education, personal and promotion). Content creator type was determined based on the evaluation of the creator’s profile and the qualifications/degrees if shared. The medical specialty of the physician posters was included if mentioned in the video or on the content creator’s page. The video category was determined based on an evaluation of the purpose behind the video. If the video was created to share a personal story, it was deemed ‘personal’; if the post aimed to educate the public, it was deemed ‘educational’; if the post was aimed to promote a product or business, it was deemed ‘promotional’. Non-English language videos were excluded from the study.
Quality assessment
We sought to analyse the videos using validated scoring metrics. Six board-certified gastroenterologists evaluated the videos with two previously validated scoring instruments: the DISCERN quality criteria for consumer health information and the Patient Education Materials Assessment Tool (PEMAT-A/V). The DISCERN instrument was designed to help health consumers judge the quality of information regarding treatment options and has been used in prior studies to assess the quality of written health information (Tam et al., 2022; Xu et al., 2021). The DISCERN instrument includes 15 different questions and an overall quality assessment with responses ranked from 1 = poor to 5 = excellent. The questions are grouped into two main categories: the first being the reliability of the publication and the second being the quality of information on treatment choices. The highest possible score on the DISCERN is 75 (DISCERN, 1997). The PEMAT-A/V was specifically designed to help determine whether patients can understand and act on audiovisual health information provided to them. The instrument contains 13 questions regarding the understandability of the content and 4 questions about the content’s actionability. The questions are scored using binary methods with 0 (disagree) or 1 (agree). Eight of the questions have a Not Applicable (NA) option. The score for each category is a percentage calculated by dividing the points received by the total possible points (Agency for Healthcare Research and Quality, 2013; Shoemaker et al., 2014). Lower scores (percentages) indicate the video may have deficiencies in the understandability and/or actionability of the video leading to the material being difficult for patients to comprehend or act on. Finally, all videos were scored for the presence of misinformation using a standard 5-point Likert-type scale. In our study, two questions from the PEMAT-A/V were omitted from the analysis (one from the understandability and one from the actionability section) due to an overwhelming number of ‘Not Applicable’ results. Specifically, these questions were (1) ‘the material uses simple tables with short and clear row and column headings’, and (2) ‘the material explains how to use the charts, graphs, tables, or diagrams to take actions’.
Ethical considerations
The Ethics Committee of the Lundquist Institute waived the need for ethics approval and patient consent for the collection, analysis and publication of the retrospectively obtained and anonymised data for this non-interventional study. According to the Lundquist Institute, neither ethics approval nor patient consent is required for registry linkage studies.
Results
Video characteristics
In total, 253 videos were reviewed with a total composite of 18.6 billion likes, 191,000 comments, 935,000 saves and 927,000 shares. #IBS was the hashtag with the greatest number of views (2.2 billion), while videos on haemorrhoids had the greatest number of likes (4.3 million), comments (59,000), and shares (288,000) (Table 1). Representative videos and comments we encountered are highlighted in the Video and Figure 1, respectively.
Engagement by hashtag.
Representative comments on Video Post.
A minority of the gastroenterology-related videos we reviewed were created by physicians 28/253 (11.1%), while 177/253 (70%) videos were created by non-healthcare workers. Of 253, 39 (15.4%) of the videos were created by non-physician healthcare workers including nurses, physical therapists and nutritionists, and the remainder were created by corporations 9/253 (3.5%). The male-to-female distribution was 72/253 (28.4%) and 174/253 (68.7%), respectively, with the rest coming from company accounts or unknown gendered persons. Among physician posters, 16/28 (57.1%) were board-certified in gastroenterology, 5/28 (17.9%) were colorectal surgeons and the rest were unspecified physicians 7/28 (25%) (Table 2). Of 253, 155 (61.3%) of the videos reviewed were created for personal reasons, 94/253 (37.1%) of the videos were created for educational purposes and 4/253 (1.6%) were created for sponsorship/corporate reasons. Among physician-created videos, 21/28 (75%) were created for educational reasons, and the rest were created for personal reasons.
Post and content creator characteristics.
Video quality assessment
Of the total 253 videos, 186 were analysed using the DISCERN and PEMAT-A/V scoring instruments, previously described in the methods section. The remaining videos were not available on the platform at the time of review. The median video quality was two out of five on the Likert-type scale with a total 23/75 (interquartile range [IQR] 15–31) DISCERN score. Most videos were rated ‘very poor’ or ‘poor’ in quality, with 114 videos (61.3%) rated as ‘very poor’ and 58 (31.2%) rated as ‘poor’. Thirteen (7%) videos were rated as ‘fair’ and only one video (0.5%) was rated as ‘excellent’ on the DISCERN grading system. The median video source quality was one out of five with 153 of the 186 (82.2%) rated as a one out of five on the clarity and quality of sources used to compile the content. The median PEMAT-A/V scores were 7/12 (73.9%) for understandability and 1/3 (33.3%) for actionability; 82.1% of the videos used everyday language (Table 3).
Video analytics.
Discussion
Short video social media platforms are rapidly rising in popularity; TikTok, specifically, has over 1 billion monthly active users with 770 million downloads worldwide in the first three-quarters of 2023 alone (Ruby, 2024). Among this growing user base, we found substantial interest in common gastrointestinal disorders, as evidenced by the high degree of engagement on social media. In this study, we identified 2.2 billion views on the hashtag ‘IBS’ alone. However, we found that the overall quality of the information in popular gastroenterology-related videos is low and that most of the popular TikTok posts within the field of gastroenterology, accounting for substantial ‘views’ and ‘likes’, are created by non-physicians.
TikTok, like many other social media platforms, is primarily geared towards user entertainment with catchy music and attractive graphics. The high-quality educational video content may lack the captivating aspects that make the posts popular among users. Prior studies have demonstrated that some of the most viewed videos related to healthcare were those promoting natural health products as opposed to primarily educational content posted by professional health societies (Adhikari et al., 2016; Lenczowski and Dahiya, 2018). One dermatological study evaluated the most viewed videos on YouTube after searching the term psoriasis treatment; two of the four top results were self-promotional videos on ‘natural cures’ promoted by reported ‘natural health specialists’ lacking accreditation (Lenczowski and Dahiya, 2018). Similarly, a gynaecologic evaluation of engagement with YouTube videos based on the search term cervical cancer identified a majority of public interest aimed at personal experience with the disease rather than an objective, educational perspective (Adhikari et al., 2016). In addition, while these two studies focused on the quality of medical content on YouTube in the fields of dermatology and gynaecology, both came to similar conclusions as our study. They found that most of the content surrounding psoriasis treatment and cervical cancer on YouTube was low in quality or did not present information based on validated research. Both studies emphasised the need for reputable medical sources and professional medical societies to contribute higher quality content on YouTube to better inform patients (Adhikari et al., 2016; Lenczowski and Dahiya, 2018). We found similar patterns within gastroenterology-related content.
The 2021 Physician Specialty Data Report lists 15,678 active gastroenterologists in the USA, compared to an estimated 1.1 million content creators on TikTok (American Association of Medical Colleges, 2021; Exploding Topics, 2023). This imbalance may contribute to a drowning out of expert opinion by potential misinformation directed towards the general population. For this reason, in addition to discerning the purpose of a post, platform users must now correctly identify the content uploader before pursuing advice or gathering clinical information. Our study demonstrated the challenges social media users face in accessing good quality, evidence-based information.
We also recognise the potential for social media to facilitate widespread change in how medical information is distributed and discussed. Several publications have noted digital media’s ability to bridge the gaps created by low socioeconomic status and health literacy. In the wake of COVID-19, the US Centers for Disease Control constructed dozens of shareable widgets, videos and graphics containing relevant public health information designed to be spread on sites like Facebook and Twitter (CDC Archives, 2023). In the field of dermatology, social media has been shown to directly increase exposure, downloads and engagement with peer-reviewed health information when compared to articles not available on similar sites (Liakos et al., 2021). High-quality educational content related to gastrointestinal disorders, although very limited, was seen in our video cohort. The one video that scored ‘excellent’ with the DISCERN tool was an educational video about haemorrhoids posted by a gastroenterology nurse practitioner. This video creator described the incidence of haemorrhoids, depicted the pathophysiology using captive drawings, and outlined the various conservative and invasive treatment options available to patients. Such videos highlight the potential benefits of social media that could be harnessed in the field of gastroenterology.
The high degree of engagement we identified with videos focusing on common gastrointestinal disorders suggests they are an important topic of discussion in today’s cultural atmosphere. Our data potentially highlight the public’s desire to explore and learn more about these conditions. Recently, TikTok has seen an increase in content creators sharing their medical histories as a way to advocate for others. One such creator, a 38-year-old woman recently diagnosed with stage IV colorectal cancer, used her story to encourage others to adhere to recommended screening guidelines (UCLA Health, 2023). Given the increased engagement, providing high-quality, evidence-based guidance to the field of gastroenterology has the potential to make a substantial impact. However, the personal content videos we examined in our video cohort were of significantly low quality.
Limitations
There are certain limitations to our analysis. Although used in prior studies to evaluate the content on social media platforms, the DISCERN and PEMAT-A/V tools have not been validated to assess the quality of social media video posts. Due to a small number of videos created by healthcare workers in our study, we could not evaluate whether physician or healthcare worker–created videos scored higher than non-healthcare worker–created content. DISCERN and PEMAT-A/V tools could not be applied to all the videos in the initial video cohort, as some videos were unavailable on the platform at the time of analysis. However, given the trend in reliability and sourcing, and the large number of videos examined, the unavailable videos are unlikely to have significantly altered the reported quality, understandability and actionability ratings.
Conclusion
This study demonstrates substantial public interest in gastrointestinal disorders on social media platforms utilising short video formats such as TikTok, as evidenced by the number of views on the most popular posts. We found that the information in gastroenterology-related videos favoured by users was neither accurate nor reliable. Given the current robust popularity of social media, there is potential for a significant positive impact via public health promotion in this space. There is a need for clinicians, professional bodies and other health professionals to develop accurate educational content for this audience and improve patient outreach and connectivity.
