Abstract
Objective
To examine patient and physician satisfaction with telemedicine in otolaryngology during COVID-19 and identify associated factors.
Study Design
Prospective cohort study.
Setting
Tertiary care center.
Methods
Patient satisfaction was rated by patients (age ≥18 years) who had encounters from May to July 2020 (n = 407). Physician satisfaction was rated by 15 otolaryngologists for specific encounters delivered from May to June 2020 (n = 1011). Patient satisfaction was measured with a Press Ganey questionnaire and a Telemedicine Satisfaction Questionnaire. Mean Press Ganey satisfaction scores of telemedicine encounters during COVID-19 were compared with the pre−COVID-19 Press Ganey scores from in-person encounters (n = 3059) to test a noninferiority hypothesis. Physician satisfaction was measured with a Provider Satisfaction Questionnaire.
Results
The mean Press Ganey patient satisfaction score for telemedicine encounters was 94.5 (SD, 8.8), no worse than that for in-person encounters prior to COVID-19 at 93.7 (SD, 15.5; Δ = 0.8 [95% CI, −0.5 to 2.1, excluding the noninferiority margin of −1]). Encounters with videoconference (vs telephone) and patients reporting higher income were associated with higher Telemedicine Satisfaction Questionnaire scores. Physician satisfaction scores during COVID-19 with telemedicine encounters were overall high at 83.3 (95% CI, 77.5-89.1), slightly lower when compared with the scores with in-person encounters at 88.4 (95% CI, 82.5-94.3; Δ = −5.2 [95% CI, −6.6 to −3.8]). Encounters with videoconference (vs telephone) and patients with English as a preferred language and follow-up visits were associated with higher Provider Satisfaction Questionnaire scores.
Conclusions
Telemedicine is a feasible alternative format in otolaryngology during COVID-19 with overall high patient and physician satisfaction. Patient satisfaction with telemedicine encounters during COVID-19 was no worse than in-person encounters prior to the pandemic. Physician satisfaction with telemedicine was relatively lower in comparison with in-person encounters.
In an institution-wide effort to minimize exposure during the COVID-19 pandemic, health care providers have transitioned a large portion of their clinical encounters to telemedicine. Successful implementation of telemedicine within otolaryngology clinics has been critical given the risk of secretion aerosolization and disease transmission during otolaryngologic encounters. 1
Despite its growing popularity and the increased need, telemedicine poses many potential challenges, such as limited capabilities for physical examination, technical difficulties, and socioeconomic barriers. Telemedicine in otolaryngology poses unique challenges as many clinical encounters rely on specialized instruments and procedures.2-4 These limitations may affect the ability to integrate telemedicine within otolaryngology while maintaining high levels of patient care and satisfaction. A few pilot studies have reported successful implementation of telemedicine in otolaryngology clinics prior to the COVID-19 pandemic.2,5-10
Recent publications describing the implementation of telemedicine within otolaryngology during the COVID-19 pandemic report mixed results. 11 Some authors note high feasibility and patient satisfaction.12,13 Others outline challenges relating to limited physical examination capabilities, compromised patient-physician interaction, technical difficulty, and poor patient perception/adoption of the telemedicine format.14-16 Very few studies have examined physician satisfaction with telemedicine in otolaryngology.
Nonetheless, the recent expansion of telemedicine encounters during the COVID-19 pandemic across all subspecialties provides an opportunity to better understand the utility of telemedicine in otolaryngology that can be applicable beyond the current pandemic. We aimed to evaluate patient and physician satisfaction with otolaryngologic telemedicine encounters during the COVID-19 pandemic in comparison with in-person encounters.
Methods
Study Design
We conducted a prospective study of patient and physician satisfaction with telemedicine encounters in the outpatient Keck Hospital otolaryngology clinic during COVID-19. This study was approved by the University of Southern California Institutional Review Board (HS-20-02853). Patient satisfaction data were collected on encounters from May 11 to July 9, 2020. Physician satisfaction data were collected on encounters from May 18 to June 25, 2020.
Eligible patients were aged >18 years who were seen for a telemedicine or in-person visit during the study period. Telemedicine encounters included visits that were conducted via videoconference or telephone. Physicians were included if they had provided at least 1 telemedicine encounter.
Patients eligible for inclusion were identified via chart review. Participants were provided an electronic link to an online REDCap survey via email within 1 week following their clinic visit; the survey was designed to evaluate the participant’s overall satisfaction with his or her clinic visit.17,18 Participant demographic data were collected (age, sex, race, preferred language, education, income, employment, and commute to clinic), as were variables relating to encounter type (new patient vs follow-up, telemedicine format, physician seen, and encounter duration).
As a comparative group, patient satisfaction data from clinical visits between January 2018 and January 2020 were queried from an existing database (n = 3059). Data were collected from the 10 care provider–related items of the Press Ganey survey and compared with responses during the study period.
Physicians eligible for inclusion in this study were recruited via email. Participants were asked to complete a 5-item questionnaire after each encounter to assess overall satisfaction with the care provided. The surveys were returned to the study personnel at the end of each clinic day. Physicians were asked to complete the questionnaire for telemedicine and in-person encounters.
Telemedicine Protocol
Videoconference encounters were conducted through the USC Telecare application. Patients registered for telemedicine encounters downloaded the USC Telecare application on a device of their choosing (smartphone, tablet, or computer) and received a link to the portal. New patients seen during the study period were given the option of videoconference, telephone, or in-person visit. Telemedicine visits were strongly recommended to minimize direct physical contact during the study period. Existing patients were prescreened and designated to a telemedicine or in-person visit at the discretion of the provider. During the study period, 88% of new encounters, 64% of follow-up encounters, and 36% of postoperative encounters were conducted with telemedicine.
Patient Satisfaction Outcomes
Patient satisfaction was evaluated with the 10 care provider–related items from the Press Ganey survey. 19 This standardized survey was designed to measure and compare patient experience and feedback, and it had been utilized for 2 years prior to COVID-19 to assess patient satisfaction in our outpatient otolaryngology clinics.
Additionally, patients’ satisfaction with their telemedicine encounter was assessed with the Telemedicine Satisfaction Questionnaire (TSQ), a validated instrument designed to assess overall patient satisfaction with the receipt of health care via telemedicine. 20 The TSQ consists of 14 items scored on a 5-point Likert scale, and it assesses 3 domains: quality of care provided, similarity to face-to-face encounter, and overall perception of the interaction. The TSQ has been utilized in prior literature among a variety of patient populations.21-23 Participants responded to an additional 2 items that we designed to assess their opinions and anxiety toward COVID-19: “How closely are you following news about the COVID-19 pandemic?” and “How worried are you that you or someone in your family will be exposed to COVID-19?” These questions were rated with a visual analog scale (0, very unlikely; 100, very likely).
Physician Satisfaction Outcomes
Physician satisfaction was assessed with the Provider Satisfaction Questionnaire (PSQ), an instrument designed to evaluate physicians’ self-rated satisfaction with the quality of the patient-physician encounter. 24 The questionnaire consists of 5 items self-rated on a visual analog scale (0, not at all; 100, extremely): patient needs addressed, patient involvement in the encounter, adequacy of information given, quality of emotional support provided, and general interaction satisfaction.
Data Analysis
Pearson chi-square test and t test were used to compare categorical and continuous variables, respectively. The standardized Press Ganey survey mean scores from the telemedicine sample were compared with 2 in-person samples: before and during COVID-19. Our primary noninferiority study hypothesis related to telemedicine satisfaction when compared with in-person encounters before COVID-19. Telemedicine Press Ganey satisfaction was deemed to be noninferior to the pre–COVID-19 Press Ganey scores if the lower 95% CI of the group difference (telemedicine minus pre–COVID-19) excluded the noninferiority margin set at –1.0. To define our noninferiority margin, we summarized previous literature reporting pre- and posttest differences in Press Ganey scores in response to interventions25-29; a sample size–weighted average of intervention differences was 3.4. We therefore identified the noninferiority margin of 1.0 a priori, which is approximately 30% of the effective difference of 3.4. A sample size of 225 telemedicine encounters were deemed to provide 80% power to test our noninferiority hypothesis.
TSQ surveys were analyzed in total mean and by 3 subdomains. Univariable and multivariable linear regression analyses were conducted to examine the relationship between demographic and clinical factors with patient TSQ and Press Ganey scores.
Physician satisfaction for each encounter was calculated by averaging the responses to the 5 items on the PSQ. Mean physician satisfaction between telemedicine and in-person encounters was compared with a linear mixed effects model. A random effect at the provider level was included to model provider-level deviation from the overall mean and to account for multiple rated encounters per physician. The relationship between patient factors and provider satisfaction was analyzed with multivariable mixed effect modeling.
All analyses were conducted with Stata 16 (StataCorp). Significance was set at P < .05 (2-tailed).
Results
The study cohort included 407 patients rating satisfaction on their visits during the study period (212 telemedicine, 195 in-person) and 15 physicians rating their satisfaction on 1011 encounters (646 telemedicine, 365 in-person). Response rates were 19% (407/2139) and 70% (1011/1443) for patient and physician satisfaction assessments, respectively. Characteristics of the encounters are summarized in Table 1 . In general, patient characteristics were similar between telemedicine and in-person encounters. Telemedicine encounters were more likely to be new encounters and with patients reporting higher income. Subspecialties in our study cohort were otology/neurotology (64 patient encounters/120 physician encounters), head and neck (95/309), laryngology (33/219), facial plastics (77/17), rhinology (96/233), and sleep/comprehensive otolaryngology (42/113).
Patient Characteristics of All Encounters With Completed Patient and Physician Satisfaction Surveys. a
Abbreviation: NA, not available.
Values are presented as No. (%) unless noted otherwise.
Mean (SD).
Mean Press Ganey patient satisfaction scores during COVID-19 remained high at 94.5 (SD, 8.8; range, 20-100) for telemedicine visits. Mean Press Ganey scores for telemedicine visits during COVID-19 were no worse than the scores for in-person visits conducted prior to COVID-19 at 93.7 (SD, 15.5) (Δ = 0.8 [95% CI, −0.5 to 2.1]; Figure 1 ). Press Ganey scores for telemedicine visits were comparable to the scores for in-person visits conducted during COVID-19 at 94.9 (SD, 10.8; Δ = −0.4 [95% CI, −2.3 to 1.5]), although noninferiority could not be concluded due to the overlap with our noninferiority margin of −1.0.
Differences in Press Ganey scores (range, 20-100) between telemedicine and in-person encounters based on noninferiority analysis. Press Ganey scores from telemedicine encounters during COVID-19 vs in-person encounters (A) prior to and (B) during COVID-19.
Patient satisfaction with telemedicine measured by the TSQ is summarized in Figure 2 . The mean TSQ score for all telemedicine encounters was high at 4.17 (SD, 0.2; range, 1-5). Mean (SD) TSQ scores were 4.2 (0.67) and 3.67 (0.83) for videoconference and telephone encounters, respectively.
Summary of patient satisfaction measured with the Telemedicine Satisfaction Questionnaire (TSQ; range, 1-5) by videoconference and telephone visits. Error bars represent 95% CI.
Factors associated with patient satisfaction among telemedicine encounters during COVID-19 were explored with 2 outcomes ( Table 2 ): Press Ganey scores and TSQ scores. In a multivariable linear regression model adjusting for relevant demographics and clinical factors, telephone visits (vs videoconference, β = −0.55 [95% CI, −0.87 to −0.23]) and older patient age (β = −0.01 [95% CI, −0.02 to −0.0005]) were significantly associated with lower TSQ scores. Higher patient income was significantly associated with higher Press Ganey scores and TSQ scores. Longer encounter duration was associated with lower Press Ganey scores.
Multivariable Linear Regression Models Exploring Factors Associated With 3 Patient Satisfaction Measures Among Telemedicine Encounters During COVID-19. a
All multivariable models were mutually adjusted and additionally adjusted for individual physician (data not shown).
P < .05.
Patients were asked, “How closely are you following news about the COVID-19 pandemic?” and graded on a 0-100 visual analog scale.
Patients were asked, “How worried are you that you or someone in your family will be exposed to the COVID-19?” and graded on a 0-100 visual analog scale.
In linear mixed effects modeling, physician satisfaction was measured with the PSQ and is summarized in Figure 3 . PSQ scores were lower among telemedicine encounters in comparison with in-person encounters (Δ = −5.2 [95% CI, −6.6 to −3.8]). PSQ scores for telephone visits were significantly lower than for videoconference visits. In a multivariable mixed effects model exploring the factors associated with physician satisfaction among telemedicine encounters, lower physician satisfaction scores were associated with patients reporting non-English as a preferred language (Δ = −7.2 [95% CI, −10.5 to −3.9]). Follow-up visits and postoperative visits were associated with higher mean PSQ scores in comparison with the new patient encounters (Δ = 3.9 [95% CI, 1.6-6.2] and Δ = 6.4 [95% CI, 1.5-11.3], respectively). Patient’s age, sex, race, and encounter duration were not associated with the physician satisfaction scores.
Summary of physician satisfaction measured with the Provider Satisfaction Questionnaire (range, 0-100). Error bars represent 95% CI.
Discussion
As telemedicine has rapidly increased in otolaryngology in response to the COVID-19 pandemic, it is important to understand patient and physician satisfaction levels toward telemedicine and their associated factors. Our study examining patient and physician satisfaction in an otolaryngology outpatient clinic during the COVID-19 pandemic demonstrates high patient and physician satisfaction with telemedicine encounters. Physician satisfaction scores were relatively lower among telemedicine encounters in comparison with in-person encounters during the COVID-19 pandemic. Non-English as a preferred language and new patient encounters were associated with lower physician satisfaction scores among telemedicine encounters.
The high patient satisfaction with telemedicine encounters in our study cohort is consistent with previous studies.30-42 It is difficult to directly compare these scores as the outcome measure, study setting, and sample size vary largely by report. 43 One study investigating patient satisfaction with telemedicine visits during COVID-19 among patients with head and neck cancer demonstrated a high mean score (6.01; range, 1-7) with the Telehealth Usability Questionnaire, but there were no comparison data with in-person visits. 41 A study based on a national database evaluating patient experience with telemedicine in otolaryngology showed slightly lower satisfaction among telemedicine visits than in-person visits. 42 However, only descriptive numbers were provided without formal statistical comparisons.
Our study is the first to demonstrate that patient satisfaction with telemedicine encounters during the pandemic was not inferior to that with in-person encounters conducted prior to COVID-19. It is important to note that the patient satisfaction data collected during the pandemic may not be generalizable to different circumstances, as the satisfaction scores may have been affected by patient attitudes toward the pandemic. The pandemic and associated restrictions may have made in-person visits less appealing, resulting in a possible overestimation of the satisfaction with telemedicine encounters. 44 Still, the majority of previous studies investigating satisfaction with telemedicine in otolaryngology were based on encounters prior to the COVID-19 pandemic and demonstrated high patient satisfaction,30-32,34-40 which have potential implications on the use of telemedicine in otolaryngology as an option beyond the current pandemic.
Telemedicine-specific satisfaction data were obtained in this cohort with the TSQ. The overall mean score of 4.2 (range, 1-5) was comparable to mean TSQ scores from studies in other medical specialties, such as diabetes care, 45 transplant surgery, 46 plastics surgery, 47 and neurology,48,49 ranging from 3.5 to 4.7.45-52 In comparison with videoconference, telephone visits had significantly lower satisfaction scores in subdomains such as quality of care and similarity to face-to-face encounters. Our results demonstrate that telemedicine with video functionality should be offered to patients whenever possible. In our cohort, videoconference encounters were unable to be performed for patients who had limited access to broadband internet, smartphone, or webcam. At least 25% of Americans are estimated to lack digital literacy skills or access to Internet-enabled digital devices to engage in videoconference visits.53,54 Therefore, current widescale implementation of telemedicine may increase disparities in health care access for vulnerable populations, such as racial/ethnic minorities, older adults, rural residents, and those with low socioeconomic status. 55 Patients with higher income reported significantly higher satisfaction with telemedicine visits, demonstrating the impact of socioeconomic status on patient satisfaction. There should be an institutional effort to ensure equitable access to telemedicine by monitoring data among patient subgroups with known limited digital literacy and taking actions to mitigate resource barriers (eg, training patients on digital skills to conduct video visits, informing patients about free or reduced-cost broadband Internet in the area). 55 Longer duration of encounter was associated with lower mean Press Ganey scores. Lower patient satisfaction may be due to longer duration of the encounter itself or unsatisfied patients taking longer. We were not able to establish a causal association in the current study.
Physician satisfaction with telemedicine was high but relatively lower than in-person encounters in this cohort. Studies prior to the COVID-19 pandemic reported a range of physician satisfaction scores with telemedicine encounters in otolaryngology.30,31,34,37,56,57 Our study is the first to compare physician satisfaction with telemedicine and in-person visits during COVID-19 in an otolaryngology outpatient setting. The relatively lower physician satisfaction with telemedicine encounters is expected given that otolaryngology relies heavily on examinations with specialized instruments and procedures. Physician satisfaction scores were also lower among telephone visits in comparison with videoconference visits. Previous studies on nonotolaryngology encounters demonstrated that the videoconference resulted in fewer medication errors, greater diagnostic accuracy, and improved decision-making accuracy when compared with telephone visits.58-60
As further implementation of telemedicine is likely inevitable during the pandemic to deliver patient care in the safest possible manner, it is pertinent to explore ways to improve physician satisfaction for telemedicine encounters. In this cohort, physician satisfaction was significantly lower when encounters were completed among patients who preferred languages other than English. Adequate interpreter access in telemedicine visits should be guaranteed to ensure improved physician satisfaction. Other innovations should be considered, such as automatically integrating staff interpreters into telemedicine encounters.
New patient encounters were more likely to have lower physician satisfaction scores in comparison with follow-up encounters for telemedicine visits. In contrast, no difference was observed in physician satisfaction between new and follow-up encounters among in-person visits. This is likely due to the limited physical examination availability to make diagnoses and management plans for new patients during telemedicine visits. Further study is needed to better understand how to appropriately triage encounters in telemedicine. As data on telemedicine visits in otolaryngology outpatient clinic expand, triaging algorithms may improve and enhance the utility of telemedicine in otolaryngology.
There are limitations in this study. Self-report of patient and physician satisfaction is inherently susceptible to participation bias and recall bias. The response rate for patient surveys was relatively low at 19%, although it falls in the range of previously reported patient satisfaction survey response rates, from 16% to 61%.61,62 It is important to interpret these results with caution as the final scores may have been affected by many aspects of participation, bias such as participants answering in socially desirable ways, repeatedly endorsing items regardless of content, or avoiding extreme response options or exaggerating their answers.63,64 In this cohort, there were no differences in response rates between in-person and telemedicine encounters. Mail and telephone surveys (which may have increased the response rate) were unable to be conducted in this study due to the limited resources and COVID-19 safety measures in place during the study period. Additionally, the current study is nonrandomized where new patients self-selected whether they would be seen via telemedicine versus in-person. Self-selection potentially influenced the range of chief complaints and may have led to higher patient satisfaction scores among patients who chose to be seen via telemedicine. This study also has the limitation of reporting experience from a single tertiary care center in an urban setting with high risk of COVID-19 exposure. Our results may not be generalizable to other health care settings with patients of different geopolitical and cultural backgrounds. Most otolaryngology subspecialties were included in our study. However, pediatric otolaryngology encounters were not available. There are many other patient and physician satisfaction questionnaires assessing various aspects of the telemedicine experiences. In this study, a set of standardized concise questionnaires were chosen to minimize survey burden. Finally, there are potential residual factors associated with patient and physician satisfaction not included in the study, such as technology self-efficacy, accessibility, familiarity with telemedicine, disease type and severity, and wait time. Detailed patient demographics were available only for patients who completed the survey. Detailed subgroup analysis by subspecialty or chief complaint was unable to be performed due to the limited number of physicians in each subspecialty and broad range of chief complaints. Future studies including clinical details such as chief complaints may enhance our understanding of telemedicine satisfaction and improve triage algorithms. Despite the major limitations, our results provide a valuable preliminary assessment of telemedicine encounters in otolaryngology outpatient clinics during the rapid adaptation of telemedicine in the setting of the COVID-19 pandemic.
Footnotes
The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.