Patient Comments about Telehealth and In-Person Care at a Federally Qualified Health Center From Patient Experience Surveys Differ in Length,Valence,and Actionability

Abstract

Background:

The COVID-19 pandemic accelerated the integration of telehealth into routine health care, especially within Federally Qualified Health Centers (FQHCs). Few studies have compared patient experiences and/or comments about telehealth versus in-person visits.

Methods:

We compared patient comments about primary care telehealth versus in-person visits. We examined 3,795 comments from patient experience surveys (1,457 telehealth and 2,338 in-person) collected electronically between April 2023 and March 2024 by a large Southern California FQHC. We coded comment valence, content, and actionability.

Results:

Patient comments about telehealth visits were shorter (mean = 13 words) than comments about in-person visits (mean = 30 words) and more positive (85% versus 81%, respectively). Comments about telehealth visits were also less likely to mention specific provider or staff behaviors, resulting in fewer actionable comments: 7% of comments about telehealth and 12% about in-person visits were deemed actionable. Most comments about telehealth and in-person visits focused on overall visit experience, rather than specific aspects of care. References to care setting and staff roles were minimal, particularly in telehealth comments.

Conclusion:

While telehealth visits are generally well-received, patient comments about telehealth provide less detail and contain less actionable information compared to comments about in-person visits. The proportion of actionable comments was considerably lower than prior studies (∼30%). This may pose challenges to using comments to improve telehealth. Understanding differences in how patients respond to the same open-ended patient experience survey question about their telehealth versus in-person visit is critical for optimizing telehealth care delivery and addressing patient needs in under-resourced settings.

Keywords

telemedicine telehealth in-person visits comments patient experience

Introduction

The COVID-19 pandemic fundamentally transformed health care delivery, most notably accelerating the integration of telehealth into routine primary care.^1,2 Federally Qualified Health Centers (FQHCs), which serve diverse and often underserved populations, adopted telehealth rapidly to maintain access to care, though they continue to face challenges delivering telehealth services.^3–5 Evidence has shown that patients at FQHCs view telehealth as a convenient way to access health care, yet patients also reported still experiencing barriers to accessing care.⁶

Understanding patient care experiences, including access and care delivery, is critical for ensuring high-quality care and for identifying areas needing improvement. Patient experience surveys, such as the Consumer Assessment of Healthcare Provider and Systems (CAHPS®) surveys, offer quantitative measures on specific domains of patient experience and include an open-ended question for qualitative feedback. Such comment data have been shown across countries and care settings to provide rich insights into patient care experiences, identifying potential issues with care and providing actionable feedback for quality improvement (QI).^7–15 Studies have consistently shown that narrative comment data are associated with the quantitative measures, provide contextually actionable guidance for improvement,^8,16,17 and can detect changes from QI efforts to improve patient-provider communication.¹⁸

While telehealth is remaining common practice for urgent and primary care, relatively little is known about how patient care experiences via a telehealth visit compare to traditional in-person visits, particularly within FQHC settings. In one quantitative study of 58,509 patient experience surveys completed after outpatient visits in a large integrated health system, patients reported equally or more positive experiences with telehealth compared to in-person care.¹⁹ Another study of 34,824 online physician reviews by patients found that while patients value clear communication in both telehealth and in-person visits, patients emphasize different factors when assessing in-person visits (i.e., office environment and wait times) versus telehealth video visits (i.e., technical issues, appointment scheduling, follow-up problems).²⁰

As telehealth becomes more established and remains available as a mode of care, studies are needed to understand whether patient comments from patient experience surveys about telehealth visits are similarly able to identify actionable guidance for improvement of virtual care. We examined and coded narrative comments in response to the same open-ended question included on patient experience surveys administered after telehealth and in-person visits and compared comments about in-person and telehealth visits.

Methods

We partnered with a large, urban FQHC in Southern California with predominantly Hispanic and Spanish-preferring patients. The FQHC collects patient experience surveys electronically, with the survey requests sent first via text message to cell phone numbers and then via email to email addresses, both containing a link to a web survey. Invitations are sent in both English and Spanish after adult and pediatric visits. Patients who had in-person visits at one of the 44 clinics completed the CAHPS Clinician and Group (CG-CAHPS) survey, whereas patients utilizing telehealth completed a 14-item non-CAHPS patient experience survey that included demographic questions about patient age and gender. Both surveys contained the same open-ended final question, “Given the coronavirus, or COVID-19 pandemic, please provide feedback on how well the provider and clinic team addressed your concerns during your [telephone/video] visit.”

We obtained patient experience survey data for visits between April 1, 2023, and March 31, 2024. Due to an administrative error, comments on the Spanish version of the telehealth survey were not captured during this same period and are not available for analysis. Overall, 4,075 adult surveys had a narrative comment: 2,483 comments from in-person visits and 1,592 comments from telehealth visits. Thirteen comments about in-person visits from the English version of the CG-CAHPS survey were in Spanish and translated into English for analysis. We excluded 280 comments that were deemed uninterpretable (e.g., “did not have one”) or contained no substantive content (e.g., “Thank you”), yielding 3,795 comments for analysis: 1,457 about telehealth visits (all in English) and 2,338 about in-person visits (1,140 in English, 1,198 in Spanish).

We developed a set of a priori codes based on prior work on coding comments from patient experience surveys.^7,21,22 These codes included codes for comment valence (positive [i.e., nothing negative was discussed], negative [i.e., nothing positive was discussed], or mixed [i.e., at least one positive and one negative comment discussed]), type of staff mentioned (e.g., provider/doctor, nurse, medical assistant, team), setting (e.g., waiting room, clinic room, after visit, telehealth), patient experience content mentioned, and actionability of comment.⁷ The content codes were applied a priori based on domains included in the patient experience surveys, e.g., access, communication, and provider rating. For actionability, if the comment contained enough specific information about “who, what, when, where, why, and how” that a health system could identify an area to highlight, i.e., an area that either needs improvement to address an issue or is an area to highlighting positive experiences, it is deemed actionable. If a comment is deemed actionable, we also coded for whom the comment is actionable, that is, whether that action would be taken with an individual provider, a set of providers and staff, such as front-office or back-office staff, and/or at an organizational level, such as across the FQHC or with an external organization, e.g., a pharmacy or specialty office.

Our four-person coding team met regularly to discuss codes, identify discrepancies, refine codes, and reach consensus. After initial code training, we obtained a pooled kappa of 0.80, indicating “very good” agreement.^23,24 Each coder was assigned comments, and we double-coded a set of 300 comments (i.e., each coder coded 150 comments so that each two-person coding pair coded 50 of the same comments). Pairwise kappas were calculated and ranged from 0.72 to 0.87, with an average of 0.80. Each coder was further assigned a set of the remaining comments to code independently. We discussed emerging codes and subcodes and clarified the use of existing codes. Once coding was completed, we calculated descriptive statistics (means and counts) of the coded comments.

We compared the presence of codes by whether the comment was from an in-person versus telehealth visit. Since we only had comments about telehealth visits from the English versions of the surveys (i.e., no Spanish telehealth comments), we compared comments from the English-language surveys across both types of visits and used z-tests for statistical significance. We also compared comments about in-person visits by survey language (i.e., comments from the English-language versus Spanish-language version of the CG-CAHPS survey). Lastly, we compared the descriptive statistics on the presence of codes about in-person visits (overall and by survey language) to previous evidence on coded comments about in-person primary care visits at the same FQHC.¹⁸ The study, its methodology, and study protocols were approved by our Human Subjects Protection Committee (IRB_Assurance No.:FWA00003425;IRB_No.:IRB00000051;Project ID:2023-N0018).

Results

Patients who had telehealth visits and completed an English-language survey were on average 52 years old and 64% female, whereas patients who had in-person visits and completed an English-language survey were on average 53 years old and 59% female (Table 1). Patients who had in-person visits and completed the Spanish-language survey were slightly older (i.e., on average 57 years old) and more likely to be female (57% female). Overall, patients who had in-person visits were 79% Hispanic, 9% non-Hispanic White, 5% Asian/Pacific Islander, 3% Black, and 3% other race. Not surprisingly, 97% of patients who had in-person visits and completed the Spanish-language survey were Hispanic, compared to 60% of patients who had in-person visits and completed the English-language version of the survey.

Table 1.

Patient Characteristics, By Visit Type and Survey Language for In-Person Visits

PATIENT CHARACTERISTICS	TELEHEALTH VISIT(COMMENTS FROM ENGLISH LANGUAGE SURVEY)(n = 1,457)	IN-PERSON VISIT(COMMENTS FROM ENGLISH LANGUAGE SURVEY)(n = 1,140)	IN-PERSON VISIT(COMMENTS FROM SPANISH LANGUAGE SURVEY)(n = 1,198)	IN-PERSON VISIT(COMMENTS FROM BOTH ENGLISH AND SPANISH LANGUAGE SURVEYS)(n = 2,338)
Age (years, mean)	51.9	52.8	56.6	54.8
Female (vs. male)	63.6%	59.1%	57.0%	58.0%
Race and Ethnicity
Hispanic	—	59.6%	97.0%	78.7%
Non-Hispanic White	—	18.2%	0.9%	9.3%
Non-Hispanic Asian/Pacific Islander	—	11.0%	0.1%	5.4%
Non-Hispanic Black	—	5.7%	0%	2.8%
Non-Hispanic Other Race	—	4.0%	1.2%	3.0%
Missing	100%	0.6%	0.8%	0.7%
Education
Less than 8^th grade	—	2.2%	26.6%	14.7%
Some high school	—	9.6%	24.1%	17.1%
High school graduate	—	24.6%	30.1%	27.4%
Some college	—	38.4%	9.9%	23.8%
College degree	—	13.2%	2.9	8.0%
More than college	—	11.0%	2.8%	6.8%
Missing	100%	0.8%	3.6%	2.3%
Self-rated overall health
Excellent	—	14.0%	14.4%	14.2%
Very good	—	22.0%	18.2%	20.1%
Good	—	37.5%	35.8%	36.6%
Fair	—	18.9%	28.1%	23.7%
Poor	—	7.2%	3.2%	5.1%
Missing	100%	0.4%	0.3%	0.3%
Self-rated mental or emotional health
Excellent	—	26.8%	31.3%	29.1%
Very good	—	25.3%	27.8%	26.6%
Good	—	25.7%	26.2%	26.0%
Fair	—	15.3%	13.2%	14.2%
Poor	—	6.7%	1.2%	3.8%
Missing	100%	0.4%	0.3%	0.3%

Comment Content

Comments from English-language surveys for in-person visits were more than twice as long as for patients who had telehealth visits, averaging 30 words compared to 13 words, respectively (Table 2). Additionally, comments about in-person visits from the Spanish-language survey were on average 18 words.

Table 2.

Comment Characteristics and Content, By Visit Type and Survey Language for In-Person Visits

COMMENT CHARACTERISTICS AND CONTENT	TELEHEALTH VISIT(COMMENTS FROM ENGLISH LANGUAGE SURVEY)(n = 1,457)	IN-PERSON VISIT(COMMENTS FROM ENGLISH LANGUAGE SURVEY)(n = 1,140)	IN-PERSON VISIT(COMMENTS FROM SPANISH LANGUAGE SURVEY)(n = 1,198)	IN-PERSON VISIT(COMMENTS FROM BOTH ENGLISH AND SPANISH LANGUAGE SURVEYS)(n = 2,338)	p-VALUE(COMPARING TELEHEALTH VISITS TO IN-PERSON ENGLISH)
Word count per comment (mean)	13.3	29.7	17.9	23.7	<0.001
Valence (i.e., Tone)
Positive only	85.1%	81.0%	87.0%	84.0%	0.005
Negative only	8.1%	8.0%	3.3%	5.6%	0.914
Mixed (Positive and Negative)	6.8%	11.1%	9.8%	10.4%	<0.001
Staff Involved
Provider/Doctor (incl. NPs, PAs, DO, Dentist)	16.7%	33.8%	23.4%	28.4%	<0.001
“Everyone”	11.6%	13.5%	17.9%	15.7%	0.143
Office staff	3.2%	8.2%	7.2%	7.7%	<0.001
Clinical team	3.6%	4.9%	3.5%	4.2%	0.108
Nurse	1.8%	6.2%	4.4%	5.3%	<0.001
Receptionist	0.7%	3.2%	3.8%	3.5%	<0.001
Staff not mentioned	69.5%	49.9%	56.3%	53.2%	<0.001
Content
Global rating plus other content	68.6%	64.8%	76.8%	71.0%	0.044
Only global rating	46.7%	33.8%	42.5%	38.2%	<0.001
Provider rating plus other content	13.5%	25.7%	16.2%	20.8%	<0.001
Only provider rating	2.5%	2.1%	1.2%	1.6%	0.468
Both global rating and provider rating plus other content	7.5%	15.6%	11.7%	13.6%	<0.001
Only global rating and provider rating	4.4%	7.1%	5.9%	6.5%	0.003
Actionability+
Not actionable	93.5%	88.3%	95.8%	92.2%	<0.001
Actionable	6.5%	11.7%	4.2%	7.8%	<0.001
Actionable for clinician/provider	3.6%	6.2%	1.6%	3.8%	0.002
Actionable for organization	1.9%	3.2%	1.6%	2.4%	0.023
Actionable for front office staff	1.9%	4.7%	2.3%	3.5%	<0.001
Actionable for back-office staff	1.0%	2.6%	1.3%	2.0%	0.002
Actionable for external organization	0.3%	0.3%	0.1%	0.2%	0.956

+Actionability across rows does not sum to “Content was actionable” because comments could be actionable by more than one actor/agent.

NP, nurse practitioner; PA, physician assistant; DO, doctor of osteopathy.

Comments from English-language surveys were predominantly only positive (compared to other valence) for both telehealth and in-person visits (85% of comments about telehealth and 81% about in-person visits, p < 0.001), while comments about telehealth and in-person visits had similar proportions of negative-only comments (i.e., 8% of comments for both telehealth and in-person visits, p = 0.914). Comment about in-person visits were slightly more mixed valence (i.e., mixed valence includes both negative and positive tone; 11% of comments about in-person visits vs. 7% about telehealth visits, p < 0.001). Notably, comments from the Spanish-language version of the in-person survey were more positive (87%), less negative only (3%), and 10% mixed.

Many more comments from English-language surveys about telehealth visits did not explicitly mention staff compared to in-person visits, 70% versus 50%, respectively. When staff were mentioned in a comment, provider/doctors were mentioned more often for in-person visits (34%) than telehealth visits (17%, p < 0.001) and mentioned in a quarter (24%) of comments from the Spanish-language surveys about in-person visits. Mentions of office staff were the next most common, but relatively few overall (8% for in-person vs. 3% for telehealth, p < 0.001), as were mentions of clinical teams (5% vs. 4%, p = 0.108, respectively) and of nurses (6% vs. 2%, p < 0.001, respectively). References to specific settings, such as waiting or exam rooms, were rare, comprising less than 1% of all comments, and only 6% of telehealth comments explicitly referred to telehealth itself. Finally, the proportion of comments from English-language surveys deemed actionable was 7% for telehealth and 12% for in-person visits (p < 0.001); the proportion of actionable comments from the Spanish-language survey about in-person visits was lower (4%).

Most comments focused on the overall experience of the visit, with 69% of telehealth comments and 65% of in-person comments expressing this (p = 0.044), and notably, about 47% of telehealth comments contained only an overall experience comment, compared to 34% on the in-person survey (p < 0.001). Comments from Spanish-language surveys about in-person visits were also more skewed toward containing a global rating plus other content (77%) or only a global rating (43%). Comments mentioning an overall rating of the provider were less frequent, contained within 14% of telehealth comments and 26% of in-person comments (p < 0.001). For comments from Spanish-language surveys, only 1% mentioned only a provider rating, and 17% mentioned a provider rating and other content. We provide examples of comments addressing each of these topics (Table 3).

Table 3.

Themes and Exemplar Quotes for Comments about Telehealth Visits and In-Person Visits

Two common themes were similarly mentioned about telehealth and in-person visits. This included comments about unspecified staff professionalism, which was referenced in 5% of both telehealth (5%) and in-person (5%, p = 0.992), and comments that mentioned the provider explained care well (4% telehealth vs. 6% in-person, p = 0.062). Other common themes that were mentioned disproportionately about telehealth and in-person visits involved staff friendliness and professionalism. For example, comments from English-language surveys referenced the provider being kind or friendly (3% telehealth vs. 8% in-person, p < 0.001), the professionalism of the provider (3% telehealth vs. 7% in-person, p < 0.001), and staff friendliness was noted without specifying the type of staff (4% telehealth vs. 6% in-person, p = 0.025). Differences were also found across telehealth and in-person visits in comments mentioning the provider showing concern (4% telehealth vs. 8% in-person, p < 0.001) and the provider listening carefully (3% telehealth vs. 6% in-person, p < 0.001). Safety and infection control measures (e.g., mask-wearing, handwashing) were also referenced more often related to in-person visits (2% telehealth vs. 9% in-person, p < 0.001), as was preventive care, mainly related to COVID-19 and flu vaccinations (2% telehealth vs. 5% in-person, p < 0.001).

Nonprovider-related content from the comments from the English-language surveys included similar mentions of ease of scheduling (4% telehealth vs. 5% in-person, p = 0.262) and overall wait times (3% telehealth vs. 4% in-person, p = 0.088) and only differed concerning whether receptionists being friendly or helpful (0% telehealth vs. 2% in-person, p < 0.001).

Comments from Spanish language surveys commonly addressed whether an unspecified provider was friendly (8%), professional (5%), listened carefully (4%), explained care (3%), or spent enough time with you (2%). They also included discussions of whether a medical provider explained care (3%), listened carefully (3%), or was professional (5%). Nonprovider-related content that was discussed most often was preventative care (6%), safety and infection control (5%), receptionists being friendly (2%), and overall wait times (2%).

Discussion

Patients completing English-language patient experience surveys were generally less likely to comment on specific aspects of their care during telehealth visits compared to in-person visits. Instead, comments about telehealth visits were more likely to describe the overall visit as positive with shorter and less detailed comments than in-person visit comments. This aligns with previous findings that shorter comments tend to be more positive yet provide less specific and actionable information about the care experience.²¹ Notably, telehealth comments less frequently mentioned provider behaviors such as showing respect or other interpersonal qualities, suggesting these are important behaviors to include in training for providers conducting telehealth visits and to address through QI efforts. Notably, patients responding to the Spanish-language surveys about in-person visits generally wrote shorter comments, as typically Spanish tends to use more words (∼25%) to convey similar concepts.

Compared to evidence on the content and actionability of comments about in-person visits on the CG-CAHPS,¹⁸ our study’s comments were similar in length to comments about in-person visits. However, our study’s valence of comments about in-person visits was more positive (85% positive versus 67% positive in previous research) with the expected lower occurrence of negative and mixed remarks. Staff were mentioned less frequently, and a much larger proportion of comments omitted any staff references (70% in telehealth and 53% in-person, compared to just 29% previously). While only about a quarter of comments mentioned providers/doctors, more than half (54%) did so in the prior analysis.

Also, our comments about in-person visits contained much less actionable content, with 12% of comments from the English-language surveys deemed actionable overall, contrasting with 39% of comments deemed actionable in the earlier study. Comments also focused more on the overall visit experience, with about 65% of comments mentioning global ratings, whereas previous evidence had roughly half of comments mentioning global ratings (35%). Importantly, we found that nearly half of comments about telehealth visits (47%) contained only mention of their overall experience, which is not actionable for making improvements. Comments including provider ratings were also less frequent (14% telehealth, 26% in-person) than previous evidence (29%). Nonprovider topics like scheduling, wait times, and receptionist interactions were mentioned less frequently than before, whereas mentions of safety procedures and preventive care were now captured, reflecting new concerns likely tied to the COVID-19 pandemic context.

Overall, compared to previous evidence about coded CG-CAHPS comments, our study revealed a higher proportion of positive but shorter comments with less substantive detail, possibly reflecting a difference in how patients value telehealth versus in-person care.¹⁸ Past research has shown that patients are less willing to pay for telehealth,²⁵ which could mean that patients do not value telehealth as much as an in-person appointment. Some patients expressed concerns in their narrative comments that providers paid less attention during telehealth visits, whereas other patients noted that providers were more attentive and present, particularly through video. These findings contrast with previous research that suggested that some patients feel providers paid less attention during telehealth visits compared to in-person visits²⁶ and another analysis of telehealth comments in which patients were sometimes not happy with the rushed nature of the provider’s approach to the visit (their “webside manner”).²⁷ Additionally, when a patient completes a telehealth visit, they likely have less to write about compared to an in-person visit. For an in-person visit, the patient must travel to the clinic, check in, wait, likely be seen by multiple providers (medical assistant, nurse, provider, and pharmacy), and then check out, compared to a telehealth visit in which the patient logs on, may wait in a virtual room, and then sees a provider, giving the patient less to write about.

Patients could also choose telehealth to address simpler, more routine problems or management of chronic conditions,^28,29 making visits shorter and giving less reason for them to write in-depth comments. Some patients have more negative perceptions of telehealth and may be less likely to choose it³⁰ If only patients who have positive opinions of telehealth participate in these visits, they may be less likely to write negative comments on the survey.

Our work has limitations. The FQHC employs about 30% bilingual physicians and has good infrastructure for and regular use of professional interpreter services (via phone and video) for both in-person and telehealth visits. However, we could not compare comments from Spanish-language surveys across telehealth and in-person visits, since comment data were not collected on the Spanish-language surveys about telehealth visits. These comments may be substantially different from the comments from English-language surveys about telehealth for several reasons. There may be cultural differences in perceptions of care experiences and quality for telehealth, as we found differences in comments from English-language surveys and Spanish-language surveys about in-person visits. Future studies could use matched or standardized analysis approaches to ensure surveys are more directly comparable. Also, if an interpreter is present on the telehealth visit, patients may want to comment on those aspects of their experience or experiences with language support. Furthermore, our comment data were collected from a single large FQHC serving a diverse and underserved population in Southern California, which may limit its generalizability to other health care settings or populations. We also had limited demographic information for the telehealth survey respondents, meaning we could only compare the similarity of the populations based on age and gender so the telehealth patient respondents may differ in unmeasured characteristics from those providing comment on the in-person survey.

To enhance the utility of telehealth patient comments for QI, health care systems might consider implementing structured prompts within surveys to encourage patients to provide more detailed feedback. Prior work has shown that such probing increases comment length and the actionable content of patient responses.²¹ Additionally, targeted interventions could focus on provider-patient communication behaviors such as respect, attentiveness, and clear explanations that appear less frequently in telehealth comments but are critical to positive patient experience.

Authors’ Contributions

Z.P. managed the coding and analysis of the comment data; analyzed and interpreted the coded comment data; drafted the article; and was involved in the final approval; N.Q. analyzed and interpreted the coded comment data; revised the article; and was involved in the final approval; D.S. analyzed and interpreted the coded comment data; revised the article; and was involved in the final approval; and D.D.Q. conceptualized and designed the study; gained funding for the study; led the acquisition of patient survey comment data; led the coding of the comment data; led the analysis of the comment data; analyzed and interpreted the coded data; drafted the article; revised the article critically for important intellectual content; and was involved in final approval.

Footnotes

Acknowledgments

The authors acknowledge the patients and families who completed surveys regarding their telehealth and in-person primary care experiences; Jimmy Gutierrez and Michelle Huerta, who assisted with obtaining the comment data; Mary Ellen Slaughter, for linking patient demographic information to the coded comment data; and Lynn Polite’s assistance with preparation of this article. This article had not been presented at a meeting.

Funding Information

This study was funded by a cooperative agreement from the Agency for Healthcare Research and Quality (AHRQ) [Contract number U18HS029321].

Data Availability

Data sharing is not applicable for this article as confidentiality was given the interviewees who participated in the study.

Ethics Approval

The study, its methodology, and all study protocols were approved by RAND’s Human Subjects Protection Committee (IRB Assurance No.:FWA00003425;IRB_No.:IRB00000051;Project ID:2023-N0018).

Consent to Participate/Publish

Passive informed consent was obtained from all individual participants included in the study as part of the process of survey completion.

Disclosure Statement

No competing financial interests exist.

References

Betancourt

, Rosenberg

, Zevallos

, et al. The impact of COVID-19 on telemedicine utilization across multiple service lines in the United States. Healthcare (Basel), 2020; 8(4):380; doi: 10.3390/healthcare8040380

Shaver

. The state of telehealth before and after the COVID-19 pandemic. Prim Care, 2022; 49(4):517–530; doi: 10.1016/j.pop.2022.04.002

Dapkins

. Telehealth and federally qualified health centers—A vulnerable future. JAMA Netw Open, 2025; 8(2):e2459510; doi: 10.1001/jamanetworkopen.2024.59510

Porteny

, Brophy

, Burroughs

. Experiences of telehealth reimbursement policies in federally qualified health centers. JAMA Netw Open, 2025; 8(2):e2459554; doi: 10.1001/jamanetworkopen.2024.59554

Weber

, Miller

, Shroff

, et al. Recent telehealth utilization at a large federally qualified health center system: Evidence of disparities even within telehealth modalities. Telemed J E Health, 2023; 29(11):1601–1612; doi: 10.1089/tmj.2022.0511

Pack

, Rusca

, Llaneza

, et al. Optimizing telehealth services: A mixed-methods needs assessment conducted among community health center patients. Med Care, 2024; 62(1):30–36; doi: 10.1097/MLR.0000000000001932

Quigley

, Predmore

. What parents have to say: Content and actionability of narrative comments from Child HCAHPS survey. Hosp Pediatr, 2022; 12(2):205–219; doi: 10.1542/hpeds.2021-006032

Quigley

, Elliott

, Slaughter

, et al. Narrative comments about pediatric inpatient experiences yield substantial information beyond answers to closed-ended CAHPS survey questions. J Pediatr Nurs, 2024; 76:e126–e131; doi: 10.1016/j.pedn.2024.02.016

Bussmann

, Muders

, Zahrt-Omar

, et al. Improving end-of-life care in hospitals: A qualitative analysis of bereaved families’ experiences and suggestions. Am J Hosp Palliat Care, 2015; 32(1):44–51; doi: 10.1177/1049909113512718

10.

Corner

, Wagland

, Glaser

, et al. Qualitative analysis of patients’ feedback from a PROMs survey of cancer patients in England. BMJ Open, 2013; 3(4):e002316; doi: 10.1136/bmjopen-2012-002316

11.

Grob

, Schlesinger

, Barre

, et al. What words convey: The potential for patient narratives to inform quality improvement. Milbank Q, 2019; 97(1):176–227; doi: 10.1111/1468-0009.12374

12.

Huppertz

, Smith

. The value of patients’ handwritten comments on HCAHPS surveys. J Healthc Manag, 2014; 59(1):31–47.

13.

Iversen

, Bjertnæs

ØA

, Skudal

. Patient evaluation of hospital outcomes: An analysis of open-ended comments from extreme clusters in a national survey. BMJ Open, 2014; 4(5):e004848; doi: 10.1136/bmjopen-2014-004848

14.

Martino

, Shaller

, Schlesinger

, et al. CAHPS and comments: How closed-ended survey questions and narrative accounts interact in the assessment of patient experience. J Patient Exp, 2017; 4(1):37–45; doi: 10.1177/2374373516685940

15.

Tsianakas

, Maben

, Wiseman

, et al. Using patients’ experiences to identify priorities for quality improvement in breast cancer care: Patient narratives, surveys or both? BMC Health Serv Res, 2012; 12(1):271; doi: 10.1186/1472-6963-12-271

16.

Quigley

, Tolpadi

, Schlang

, et al. How closed-ended survey questions and narrative comments interact in characterizing caregivers’ overall assessment of hospice care. Am J Hosp Palliat Med, 2026; 4(1):410–422.

17.

Kincaid

. Improving consumer assessment of health care providers and systems communication scores in hospice care: A pilot project. J Hosp Palliat Nurs, 2020; 22(5):359–362; doi: 10.1097/NJH.0000000000000676

18.

Quigley

, Predmore

, Martino

, et al. Patient comments on the consumer assessment of healthcare providers and systems clinician and group (CG-CAHPS) survey reflect improvements in provider behaviors from coaching. J Healthc Manag, 2023; 68(4):251–267; doi: 10.1097/JHM-D-22-00140

19.

Hays

, Skootsky

. Patient experience with in-person and telehealth visits before and during the COVID-19 pandemic at a large integrated health system in the United States. J Gen Intern Med, 2022; 37(4):847–852; doi: 10.1007/s11606-021-07196-4

20.

Chandrasekaran

, Bapat

, Jeripity Venkata

, et al. Do patients assess physicians differently in video visits as compared with in-person visits? Insights from text-mining online physician reviews. Telemed J E Health, 2023; 29(10):1557–1565; doi: 10.1089/tmj.2022.0507

21.

Quigley

, Predmore

. Parents have more to say: Comments from the child HCAHPS single question versus a narrative item set. Hosp Pediatr, 2023; 13(4):345–356; doi: 10.1542/hpeds.2022-007046

22.

Quigley

, Slaughter

, Gidengil

, et al. Usefulness of child HCAHPS survey data for improving inpatient pediatric care experiences. Hosp Pediatr, 2021; 11(10):e199–e214; doi: 10.1542/hpeds.2020-004283

23.

Landis

, Koch

. The measurement of observer agreement for categorical data. Biometrics, 1977; 33(1):159–174.

24.

De Vries

, Elliott

, Kanouse

, et al. Using pooled kappa to summarize interrater agreement across many items. Field Methods, 2008; 20(3):272–282.

25.

Predmore

, Roth

, Breslau

, et al. Assessment of patient preferences for telehealth in post–COVID-19 pandemic health care. JAMA Netw Open, 2021; 4(12):e2136405-e2136405; doi: 10.1001/jamanetworkopen.2021.36405

26.

Andreadis

, Muellers

, Ancker

, et al. Telemedicine impact on the patient–provider relationship in primary care during the COVID-19 pandemic. Med Care, 2023; 61(Suppl 1):S83–S88; doi: 10.1097/MLR.0000000000001808

27.

Hamaker

, Hyman

, Lodaria

, et al. Understanding patients’ negative experiences with telehealth: A content analysis of survey data. J Patient Exp, 2024; 11; doi: 10.1177/23743735241240881

28.

Imlach

, McKinlay

, Middleton

, et al. Telehealth consultations in general practice during a pandemic lockdown: Survey and interviews on patient experiences and preferences. BMC Fam Pract, 2020; 21(1):269; doi: 10.1186/s12875-020-01336-1

29.

Tavernier

RLE

, Blaszkowsky

, Jacobs

, et al. Patient preferences for telemental health care in a federally qualified health center. Telemed J E Health, 2025; 31(4):483–489; doi: 10.1089/tmj.2024.0458

30.

, Andreadis

, Schpero

, et al. Patient experiences with and preferences for telemedicine relative to in-person care during the COVID-19 pandemic. Telemed J E Health, 2024; 30(1):67–76; doi: 10.1089/tmj.2022.0311

THEME/CODED CONTENT	COMMENTS ABOUT TELEHEALTH VISITS (FROM ENGLISH-LANGUAGE SURVEY)	COMMENTS ABOUT IN-PERSON VISITS (FROM ENGLISH-LANGUAGE SURVEY)	COMMENTS ABOUT IN-PERSON VISITS (FROM SPANISH-LANGUAGE SURVEY)
Ratings
Global rating	“I am happy with the service provided.” –64-year-old male patient who completed an English-language survey “Very poor … I was expecting quite a bit more than what I got.” –41-year-old female patient who completed an English-language survey	“Excellent. Each time I visit my doctor I have a good experience” –56-year-old Black female patient who completed an English-language survey“All in all, it was exceptional service all the way around. I’m so glad that I chose [FQHC NAME] as my medical provider. Very pleased” –41-year-old White female patient who completed an English-language survey	“They took good care of me. Good service, thank you.” – 62-year-old Hispanic female patient who completed a Spanish-language survey
Provider rating	“I have nothing negative to say about [DOCTOR NAME] he’s truly been heaven sent” –46-year-old male patient who completed an English-language survey“100/100 Very good [DOCTOR NAME]” –30-year-old female patient who completed an English-language survey	“[DOCTOR NAME] is my Primary Dr & she is Excellent at her Job. Keep up the Great Work.” –69-year-old White female patient who completed an English-language survey“[DOCTOR NAME] is an excellent doctor! He is always on top of my medical needs and goes above and beyond in taking care of any medical concerns that I have.” –45-year-old Hispanic female patient who completed an English-language survey	“Very happy with [DOCTOR NAME], he gives me a lot of confidence and solves all my concerns, I feel that if he is interested in my health, in fact I am already recommending the doctor and the clinic.” – 50-year-old Hispanic female patient who completed a Spanish-language survey
Experience of Provider
Provider spent enough time	“Even though I was not physically at the office, my doctor still took his time addressing my concerns and was patient and always made me feel comfortable especially with my anxiety issues” –41-year-old female patient who completed an English-language survey“Also, the doctor appointments often feel rushed, and I don’t have enough time to address all of my problems” –47-year-old female patient who completed an English-language survey	“I appreciate him for going through my chart and taking the time to go through everything so thoroughly” –23-year-old Hispanic female patient who completed an English-language survey	“A little more time from the doctor because everything is very fast” –54-year-old Hispanic male patient who completed a Spanish-language survey
Provider explained care	“During my tele-visit, I felt that [DOCTOR NAME] displayed a thorough health overview and she explained to me what I needed to follow up with as far as preventative care goes, mammogram, colonoscopy, pelvic exam, pap smear.” –46-year-old female patient who completed an English-language survey“Everyone was so helpful from the person scheduling my appointment to the doctor. The doctor explained everything.” –43-year-old female patient who completed an English-language survey	“They explained what they were doing & why certain meds could help me” –69-year-old White female patient who completed an English-language survey“The provider I was assigned to wasn’t very understanding when I asked for him to explain some questions I had” –45-year-old Hispanic male patient who completed an English-language survey	“They only suggested the coronavirus vaccine. The doctor didn’t explain things well enough.” –53-year-old Hispanic male patient who completed a Spanish-language survey
Provider listened carefully	“I felt they actually [listen] to me describing my symptoms” –50-year-old female patient who completed an English-language survey“They have not listened to any of my concerns. They won’t help me with testing, and they won’t help me with vaccination.” –44-year-old female patient who completed an English-language survey	“She listens and notes my concerns about my health, I’ve had many doctors, but I have never had a doctor that would listen to me like [DOCTOR NAME] does” –54-year-old Hispanic male patient who completed an English-language survey“Staff and dr [did] not listen on my needs and pushed me out of room as fast as possible” –46-year-old White male patient who completed an English-language survey	“My visit with [DOCTOR NAME] my gynecologist, was very good, she took the time to listen to my questions regarding my health and gave me confidence.” — 52-year-old Hispanic female patient who completed a Spanish-language survey
Provider showed respect	“My provider and your clinic team show a lot [sic] of love and respect.” –52-year-old female patient who completed an English-language survey	“They treated me with respect” –62-year-old White female patient who completed an English-language survey	“My doctor is a respectful and kind person, a professional” –70-year-old Hispanic male patient who completed a Spanish-language survey
Provider showed concern or was caring	“Everyone was kind and courteous, and expressed interest and desire to assist with my medical problem” –39-year-old female patient who completed an English-language survey“She didn’t show no care for nothing [sic]sounded like she was mad or something” –31-year-old female patient who completed an English-language survey	“[DOCTOR NAME] was a breath of fresh air. One of the best and most caring physicians I’ve seen in a very long time” –40-year-old female patient of more than one race who completed an English-language survey“He cares about my well-being” –53-year-old Black female patient who completed an English-language survey	“The doctor’s care is excellent, he cares about my health, and I always feel supported by him, few doctors really care about their patients like he does.” – 46-year-old Hispanic female patient who completed a Spanish-language survey
Provider was professional	“The provider addressed my medical concerns with professionalism and care. Answering all my questions, the Provider provided good medical information.” –58-year-old female patient who completed an English-language survey“I felt like I was at the Doctor’s office, very professional and prompt and informative” –62-year-old female patient who completed an English-language survey	“Very professional manner” –61-year-old female patient of more than one race who completed an English-language survey“Very professional. Better then [OTHER PROVIDER ORG]” –61-year-old Hispanic male patient who completed an English-language survey	“Everyone was very friendly and professional. This was my first appointment, and I was very pleased with the professional medical care of the staff that took care of me” –57-year-old Hispanic male patient who completed a Spanish-language survey
Provider was kind	“My provider and team members were very kind, helpful and caring.” –55-year-old female patient who completed an English-language survey“Everyone was kind and courteous, and expressed interest and desire to assist with my medical problem” –39-year-old female patient who completed an English-language survey	“She was so nice and so kind to me. Made sure I was taking my meds and had all my [follow-up] appointments” –42-year-old White female patient who completed an English-language survey	“All the staff were very friendly, nurses and doctors” –63-year-old Hispanic female patient who completed a Spanish-language survey
Actionability
Not actionable	“It was addressed very well and such an easy process. I will recommend telephone/video visit to others.” –32-year-old male patient who completed an English-language survey	“Best team ever. Good experience” –28-year-old Hispanic male patient who completed an English-language survey	“Everything is fine.” –72-year-old Hispanic female patient who completed a Spanish-language survey
Actionable for provider	“I had an appointment with [DOCTOR NAME] on [DATE/TIME]. I’ve explained to him that I’ve been very sick for almost a month now all the symptoms I told him, and he diagnosed me with Bacterial Sinusitis and had told me to take antibiotic to heal and get better. He said he will place a prescription for me to the pharmacy I provided to them in my file. Today’s date is [DATE2], and I have not received any prescription, and I have called my pharmacy multiple times to tell me that [DOCTOR NAME] had never prescribed me antibiotics.” –36-year-old female patient who completed an English-language survey	“This doctor refused to give me the depo shot, saying my PC was wrong for scheduling my appointment on that day. He refused to discuss why he disagreed with what my PC was scheduling. I had already waited a long time for this appointment & when he refused to give me my depo shot, I asked to leave and not be billed for the appointment because he hadn’t done anything!! This was the only moment where it seemed like he cared about the appointment. Not about me, but the billing.” –41-year-old American Indian/Alaska Native female patient who completed an English-language survey	“He mentioned to me the importance of getting vaccinated against COVID and of taking this pandemic seriously and responsibly. I am delighted with the attention of [DOCTOR NAME], excellent person, very professional.” –63-year-old Hispanic female patient who completed a Spanish-language survey
Actionable for Federally Qualified Health Center	“I feel that it went well, only wish that I was able to have my lab results in front of me, or was directed on how to access them through the patient portal. This way I would have felt much more prepared. Was pleased to know that I had an issue with my feet. The Doctor, asked about it and was able to make an appointment with a specialist on the spot.” –56-year-old female patient who completed an English-language survey	“Good parking space and good receptionists. I just wish they could schedule the next visit before patient leaves even if it’s 3 months after so it can be calendared in patient’s schedule.” –74-year-old Asian/Pacific Islander female patient who completed an English-language survey	“My bad experience was only with the phone service it can go hours for them not to help you …! At least …!; For phone assistance, when the recording says there are no more people in front of the call, you just spend a lot of time and the recording goes on, it goes on and they can never take your call when there are no more calls before mine …” –55-year-old Hispanic female patient who completed a Spanish-language survey