Abstract
Objective:
To characterize the use of race and socioeconomic status (SES) variables in clinical otolarynogologic research.
Methods:
Databases were queried for all articles published in 2016 issues of 5 major otolaryngologic journals. One thousand, one hundred and forty of 1593 articles abstracted met inclusion criteria for analysis.
Results:
In total, 244 (21.4%) studies specified race as a variable. The subspecialty of Head and Neck cancer specified race at statistically higher rates compared to other subspecialties (P = .002). Two hundred nine (34.0%) domestic studies specified race compared to 35 (6.7%) international studies. Of the 244 studies that specified race, 79 (32.4%) defined race using racial and ethnic categories interchangeably. Two hundred twenty-four (91.8%) studies reported data by race, 145 (59.4%) analyzed the data, and 112 (45.9%) discussed race-based results.
In total, 94 (8.2%) studies specified SES. All subspecialties specified SES at statistically similar rates. Seventy (11.4%) domestic studies specified SES compared to 24 (4.6%) international studies. Of the 94 studies that specified SES, 42 (44.7%) defined SES using insurance status, 35 (37.2%) used education, and 32 (34.0%) used income. Seventy-eight (83.0%) studies reported data by SES, 71 (75.5%) analyzed the data, and 68 (72.3%) discussed SES-based results.
Conclusion:
In clinical otolaryngologic research, the study of race and SES is limited. To improve quality of research and patient care for all patients, investigators should clearly justify their use of race and SES variables, carefully select their measures of race and SES (if the use of these variables is justified), and study race/SES-based data beyond just a superficial level.
Introduction
Race and socioeconomic status (SES) significantly affect the delivery of health care services and patient outcomes. In 2003, a seminal report by the Institute of Medicine (now the National Academy of Medicine, or NAM) found that racial and ethnic disparities in healthcare not only exist, but are also often associated with socioeconomic differences and worse patient outcomes. 1 In the following years, national and international reports have continued to show that health disparities persist, prompting some much-needed research and policy initiatives at the federal level in the United States.2-6
However, in recent years, there has been newfound urgency in examining the role of race and SES within the US. In particular, the COVID-19 pandemic has shed new light on systemic barriers present within America’s medical system, with mounting evidence showing that disadvantaged populations bear a disproportionate burden of illness, disability, and premature death.4,7 Furthermore, the police killings of George Floyd and others revealed the pervasive nature of structural racism and its impact on Black Americans and other disenfranchised groups.8-10 Although the overall scope of disparities in America is much larger than the field of medicine, it has become clear that more work is needed to tackle systems of inequality and discrimination within the American health care system.9,10
This call to action applies to all specialties of medicine, and the field of otolaryngology is no exception. Within the last few years, numerous studies have compiled extensive data documenting differences in delivery or outcome of otolaryngologic care due to racial or socioeconomic factors.11-18 For example, some studies have found that patients with public insurance experienced longer delays when obtaining appointments for obstructive sleep apnea and otitis media, non-white pediatric patients were more likely to experience delays for cleft palate repair, and patients with low SES were more likely to experience complications from adenotonsillectomies.12-14 This growing body of evidence across a diverse spectrum of otolaryngologic care suggests that significant research and advocacy efforts are needed to intervene on disparities within otolaryngology. 15
Although some studies have established guidelines on how to consider race and SES variables, it is currently unclear if otolaryngologic investigators have been studying these variables in a consistent and effective manner.19-22 Thus, the objective of this study is to characterize the use of race and SES variables in published otolaryngologic research. In order to fully address this topic, this study focused on 3 separate questions: (1) if race/SES were consistently specified as variables, (2) how race/SES variables (if specified) were being operationalized (ie, how these variables were defined using other measurable variables), and (3) to what extent race/SES variables were handled by researchers.
Methods
The data were collected using a cross-sectional study design inspired by Liang et al and Mansukhani et al23,24 This study utilized deidentified, publicly available data, so BUMC IRB review was not required.
Five major otolaryngologic journals were selected, and all articles published in 2016 were queried for review by hand-browsing of these journals’ table-of-contents (Figure 1). To evaluate frequently cited clinical research, journals were selected based on impact factor (according to the Thomson-Reuters’ Journal Citation Reports) and proportion of clinical studies. 25

Study design and distribution of all articles initially included in the study.
From there, upon further review of the full text, articles were included or excluded from further analysis (Figure 1). The inclusion criteria involved articles that were original, human, clinical research. The exclusion criteria involved articles that were (1) editorials, correspondence, or some other commentary, (2) best practice guidelines, (3) economic or medicolegal studies, (4) systematic reviews or meta-analyses, or (5) technical, animal, basic science, or some other non-human research.
Data abstracted from each article included: journal and title of study, subspecialty of study, location of study, data collection method, specification of race as a variable, specification of SES as a variable, operationalization of race (if specified as a variable), operationalization of SES (if specified), handling of race (if specified), and handling of SES (if specified).
Subspecialty was reported based on an article’s content as one of the following: head and neck (H&N) cancer, laryngology, otology, pediatric, rhinology, thyroid, reconstructive (including cosmetic), speech and swallow, vestibular, sleep medicine, noncancerous salivary gland pathology (“salivary”), infectious disease, skull base surgery (“skull base”), and miscellaneous (“misc.”) for any other uncategorized studies. The location was reported as domestic or international based on the corresponding author’s mailing address (a domestic study was defined as being conducted within the United States). The data collection method was reported as either prospective (if investigators collected the original data themselves) or retrospective (if the data was acquired from medical records or a database). Articles with prospective methods were then further categorized as either RCT (randomized controlled trial) or non-RCT. The specification of race and SES variables was reported if either topic was mentioned anywhere within the article.
Because researchers frequently used racial and ethnic concepts interchangeably, the operationalization of race variables by a given study was reported as either (1) use of racial categories only, (2) use of ethnic categories only, (3) use of both racial and ethnic categories together as one variable, or (4) use of both racial and ethnic categories separately as 2 distinct variables. To standardize the distinctions between race and ethnicity, this study used definitions as established by the census bureau of the country in which the study took place. In addition, because researchers studied different aspects of SES, the operationalization of SES variables was reported using subcategories of socioeconomic factors that affect patient outcomes, such as “insurance status,” “education level,” and “household income.” 26
Lastly, because research varied in the extent to which race and SES variables were studied, the handling of race/SES variables was reported using 3 different categories to assess (1) if any data were reported by race or SES, (2) if there was any statistical analysis of race or SES-based data (one reported P-value was sufficient), and (3) if there was any discussion of race or SES-based results within the text.
All data were verified by multiple authors for accuracy. χ2 tests with Pearson residuals were used for statistical analysis; significance was adjusted from P < .05 with Bonferroni corrections for multiple comparisons. Microsoft Excel (Microsoft Inc., Redmond, WA) was used for analysis.
Results
In total, 1140 articles abstracted met inclusion criteria for analysis, while 453 articles were excluded based on exclusion criteria (Figure 1).
Specification of Race and Socioeconomic Status as Variables
Of these 1140 studies, 244 (21.4%) specified race as a variable while 94 (8.2%) specified SES (Figure 2). χ2 tests revealed (1) domestic studies when compared to international studies specified both race and SES variables at statistically higher rates (P < .001), (2) retrospective studies specified race variables at statistically higher rates compared to prospective studies (P < .001), and (3) RCT prospective studies and non-RCT prospective studies did not have statistically different rates of specifying race and SES variables (P > .025). (Figures 3-5).

All studies that specified race/SES variables categorized by subspecialty. Error bars denote 95% CI (sample size above error bar). * = statistically significant difference with Bonferroni corrections (P < .0038).

All studies that specified race/SES variables categorized by location (domestic vs international). Error bars denote 95% CI (sample size above error bar). ** = statistically significant difference with Bonferroni corrections (P < .025).

All studies that specified race/SES variables categorized by methodology (prospective vs retrospective). Error bars denote 95% CI (sample size above error bar). *** = statistically significant difference with Bonferroni corrections (P < .025).

Prospective studies that specified race/SES variables further subcategorized by methodology (RCT vs. non-RCT). Error bars denote 95% CI (sample size above error bar). **** = statistically significant difference with Bonferroni corrections (P < .025).
Operationalization of Race and Socioeconomic Status Variables
Of the 244 studies that specified race, 125 (51.2%) defined race with one variable using exclusively racial categories, 29 (11.9%) defined race with one variable using exclusively ethnic categories, 79 (32.4%) defined race with one variable using a combination of both racial and ethnic categories, and 11 (4.5%) defined race using 2 distinct variables: one for racial and one for ethnic categories.
Of the 94 studies that specified SES, 42 (44.7%) defined SES in terms of participant insurance status, 35 (37.2%) defined SES in terms of participant education level, 32 (34.0%) defined SES in terms of participant household income, 4 (4.3%) defined SES in terms of participant occupation, 16 (17.0%) defined SES in terms of some other variable unique to the study itself (eg, patient community health risk), and 9 (9.6%) did not operationalize SES at all. Note that several studies operationalized SES using more than one approach and thus were included in more than one of the categories.
Handling of Race and Socioeconomic Status Variables
Of the 244 studies that specified race, 224 (91.8%) reported data by race, 145 (59.4%) analyzed data on race, and 112 (45.9%) discussed race-based results (Table 1). Of the 94 studies that specified SES as a variable, 78 (83.0%) reported data by SES, 71 (75.5%) analyzed data on SES, and 68 (72.3%) discussed SES-based results (Table 2). Note that these categories were used to describe the range of research conducted by investigators and were assessed independently of each other; one study could be included in more than one of the categories.
Studies Categorized by Handling of Race-Based Data.
Studies Categorized by Handling of SES-Based Data.
Discussion
Specification of Race and Socioeconomic Status as Variables
Few studies specified race; even fewer specified SES as variables (Figure 2). Further analysis revealed that this trend persisted across different subspecialties, locations, and methodologies (Figures 2-5).
Analysis by location also revealed that domestic studies specified both race and SES variables at statistically higher rates when compared to international studies (Figure 3). These findings may be explained by the different approaches that nations use to study constructs like race and SES. 27 For example, while some countries like the United States openly encourage investigators to collect and discuss data on race, other countries like France, Spain, and Germany have strict policies that limit the collection and processing of racial and ethnic data, even for census purposes.27-29 These policies likely affect the measured rate of international studies specifying race and SES variables because research from countries with such policies make up a significant proportion of otolaryngologic literature.27,30
Additional analysis by methodology revealed that (1) retrospective studies studied race at statistically higher rates compared to prospective studies and (2) RCT prospective studies and non-RCT prospective studies did not have significantly different rates of studying race and SES (Figures 4 and 5). These findings are notable because retrospective studies often rely on observational health databases including electronic medical records, yet these databases often contain incomplete or inaccurate data on race and SES.31,32 These findings are also notable because the FDA mandates clinical trials to report participant demographics, yet RCT prospective studies were not found to report race nor SES at statistically higher rates, and prospective studies as a whole were found to report race at statistically lower rates.
Overall, these findings suggest that the use of race and SES variables within clinical otolaryngologic research has been limited. These findings were unexpected given efforts over the years by academic institutions to promote research on these topics.5,6 That being said, it is important to note that these results describe a large, heterogenous collection of articles on a wide range of topics. Because the decision to include race and SES variables depends on the relevance of race/SES-based data to the scientific question at hand, the exclusion of race and SES variables may be reasonable within the context of one specific study, while the same decision to exclude these variables may not be defensible within the context of another study. 33
Regardless of the possible reasons for the currently limited use of race variables, it is critically important that institutions (both domestic and international) avoid overcorrecting for this phenomenon. While the most obvious and convenient solution may be to simply require researchers to incorporate race variables into all future study designs, a more nuanced approach is necessary since the improper justification for the inclusion of race variables can actually perpetuate rather than address disparities. 22 This is because racial differences found in large data sets more frequently reflect effects of racism (such as toxic stress and its physiologic consequences) rather than genetic variation between racial groups.22,34,35 In such cases, race-based adjustments of clinical algorithms and research would fail to address the actual cause of the disparity and may even risk cementing further inequity into the system by discouraging clinicians from offering clinical services to certain patients. 22
Operationalization of Race and Socioeconomic Status Variables
Investigators did not have one consistent approach to conceptualizing race variables. Notably, 32.4% of studies defined race with one variable using a combination of both racial and ethnic categories. This contrasted with a much smaller 4.5% of studies that defined race using 2 distinct variables (one specifically for racial categories and one for ethnic categories).
This finding suggests a lack of consensus about how race variables should be defined in clinical research. The lack of a standardized definition when conceptualizing race as a variable (one that consistently distinguishes between race and ethnicity) limits the generalizability of race-based conclusions drawn in research. A standardized definition of race would be beneficial for clinical research, perhaps one derived from the evidence-based racial/ethnic categories proposed by the NAM report “Standardization of Race, Ethnicity, and Language Data.”36,37
Investigators also conceptualized SES variables using several different approaches. Certain variables were more frequently used than others to define SES, including education, income, and insurance status. Most studies used a unique combination of these variables, whereas a small handful chose not to operationalize SES at all and instead studied SES in generic, non-measurable terms.
Since SES is a complex, multidimensional construct comprising a diverse array of factors (typically related to economic resources, power, and/or prestige), the observed variability in defining measures of SES was expected.1,38-40 However, to further improve the generalizability of SES-based conclusions, otolaryngologic investigators should take care to (1) systematically select measures of SES based on their relevancy to the health outcome and population of interest and (2) consider how other socioeconomic factors that were left unmeasured may still affect the research.39,40
Handling of Race and Socioeconomic Status Variables
Investigators often did not consider race/SES-based data to be significant enough for statistical analysis or discussion. Otolaryngologic studies that specified race as a variable were found to have reported race-based data at much higher rates than they analyzed and discussed race-based results (91.8% reported >59.4% analyzed >45.9% discussed). This same decreasing trend was also present for studies that specified SES, although the decrease was not nearly as pronounced (83.0% reported >75.5% analyzed >72.3% discussed).
Generally speaking, these data suggest that studies that specified race as a variable typically reported race when describing participant demographic characteristics without any further analysis or discussion. The same was generally found to be true for studies that specified SES, although not to the same extent. These findings suggest that while some investigators considered race and SES variables to be important during data collection, they often did not go on to consider these factors later on in their research.
Research Implications
The current scientific consensus that there are both appropriate and inappropriate uses of race and SES variables stems from a relatively recent shift in paradigm that clinical research has the potential to perpetuate health inequities.22,33,39 Although the articles analyzed in this study were published in 2016 before this paradigm became mainstream, it is important to regularly scrutinize the research and tools used in clinical practice today for the benefit of our patients.
Altogether, findings from this study indicate that certain changes are needed to improve the use of race and SES variables. Specifically, investigators should (1) justify their use of race/SES variables, (2) use standardized measures of race (if the use of race variables is justified), (3) select relevant measures of SES in a systematic manner (if the use of SES variables is justified), and (4) study race/SES variables beyond just a superficial level (ie, analyzing and discussing race/SES-based data).
While this study specifically focused on identifying actionable changes for individual investigators to make, the literature has consistently found that a comprehensive, multi-level strategy is needed to effectively address the disparities that affect minority and socioeconomically disadvantaged populations.1,3,41 Thus, otolaryngologic institutions must also take an active role in coordinating efforts to improve the use of race and SES variables.
Limitations and Future Directions
This study has several limitations. The focus on articles published in 2016 affects the generalizability of this study’s conclusions, especially since current investigative practices have likely changed in the setting of George Floyd’s recent death and the ongoing COVID-19 pandemic. 42 The cross-sectional viewpoint of only 1 year’s worth of research also results in limited longitudinal data to substantiate the claims made above. This study was restricted in scope to articles published in highly circulated journals, so the conclusions drawn may not necessarily be representative of all published otolaryngologic research. Lastly, the generalizability of some conclusions may be limited because it (1) used census bureau definitions of race (which continue to change over time), and (2) only indirectly assessed from the literature how investigators conceptualized race/SES variables (rather than directly assessing this from investigators themselves). 43
Given these limitations, future directions should include longitudinal studies of the literature to follow the trends described in this study, with careful attention to literature published in more recent years to better characterize the most current investigative practices within the field of today. Furthermore, future research should examine more journals (especially those published internationally and by different subspecialty societies) to develop a more comprehensive and complete picture of the use of race and SES variables within otolaryngologic research. Lastly, future research should explore the use of sexual orientation and gender identity (SOGI) variables within clinical otolaryngologic research, especially since the current research has shown significant health disparities among the LGBTQIA + community, yet the uptake of SOGI data collection and documentation within clinical research has been found to be abysmally low. 44
Conclusion
Within clinical otolaryngologic research, the study of race and SES is limited. Despite efforts by academic institutions to promote research on these topics, few studies were found to have specified race and even fewer specified SES as variables. In addition, among those studies that did specify these variables, several were found to have operationalized or handled them inadequately.
To improve quality of research and patient care, especially for minority and socioeconomically disadvantaged populations, otolaryngologic investigators should (1) clearly justify their use of race and SES variables, (2) carefully select their measures of race and SES (if the use of these variables is justified), and (3) study race/SES-based data beyond just a superficial level. In addition, otolaryngologic institutions must also take an active role and coordinate efforts to improve the study of race and SES, especially since a multi-level strategy is needed to effectively address disparities within otolaryngology.
Supplemental Material
sj-png-1-aor-10.1177_00034894221111323 – Supplemental material for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research
Supplemental material, sj-png-1-aor-10.1177_00034894221111323 for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research by Yash Prakash, Libby M. Ward, Zaroug Jaleel, Vinith Ilavarasan, Jennifer J. Liang, Mihir Prakash and Jessica R. Levi in Annals of Otology, Rhinology & Laryngology
Supplemental Material
sj-png-2-aor-10.1177_00034894221111323 – Supplemental material for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research
Supplemental material, sj-png-2-aor-10.1177_00034894221111323 for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research by Yash Prakash, Libby M. Ward, Zaroug Jaleel, Vinith Ilavarasan, Jennifer J. Liang, Mihir Prakash and Jessica R. Levi in Annals of Otology, Rhinology & Laryngology
Supplemental Material
sj-png-3-aor-10.1177_00034894221111323 – Supplemental material for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research
Supplemental material, sj-png-3-aor-10.1177_00034894221111323 for The Use of Race and Socioeconomic Status Variables in Published Otolaryngologic Research by Yash Prakash, Libby M. Ward, Zaroug Jaleel, Vinith Ilavarasan, Jennifer J. Liang, Mihir Prakash and Jessica R. Levi in Annals of Otology, Rhinology & Laryngology
Footnotes
Acknowledgements
We thank Dr. Howard Cabral, PhD (Boston University School of Public Health) for his assistance with the study design for this project.
Authors’ Note
A subset of the content in this manuscript was presented as an oral presentation at the Triological Society’s 2020 Combined Sections Meeting held in Coronado, CA from January 23-25, 2020.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was supported by the National Center for Advancing Translational Sciences, National Institutes of Health, through
IRB Review
IRB review was not required because this study utilized published, deidentified, and publicly available data from online academic databases (PubMed, Wiley Online Library, etc.).
Supplemental Material
Supplemental material for this article is available online.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
