Abstract
Among 173 patients seeking musculoskeletal specialty care, we sought patient personal factors associated with patient experiences measured using the 7-item Trust and Experience with Clinicians Scale (TRECS-7) and Jefferson Scale of Patient's Perceptions of Physician Empathy (JSPPPE). Accounting for potential confounders, including demographics, visit-related information, mental health, and social health, we found no factors associated with TRECS-7, and only self-reported Hispanic/Latino ethnicity was associated with lower JSPPPE (regression coefficient = −2.8, 95% confidence interval = −4.9 to −0.63). In posthoc cluster analysis, statistical groupings of patients with generally worse mean social health and mental health were associated with worse patient experience (TRECS-7 and JSPPPE). The combination of an experience measure with lower ceiling effects and wider distribution of scores and cluster analysis may improve the ability to measure associations with patient personal factors.
Keywords
Introduction
Background
Clinicians use patient-reported experience measures (PREMs) to quantify the patient's perception of the clinician's empathy and communication effectiveness, their trust in the clinician, their willingness to recommend the physician to family or friends, and other aspects of experiences with care. 1 As one might expect, these various PREMs are highly correlated with one another.2-7 But, to date, the measured relationships between experience measures and patient personal factors are modest and inconsistent.3,8,9 This may be due to another characteristic of PREMs: they all have notable ceiling effects (a large proportion of people with the highest possible score).10,11 Ceiling effects lead to loss of information at the top end of the scale. It's possible that the loss of information related to these ceiling effects may be limiting the ability to represent variations in experience adequately and to measure relationships between levels of patient experience and specific clinician or patient factors. Similar to other large businesses and organizations, clinicians and care units can use PREMs to measure the effects of efforts to improve patient experience. 12 The ability to improve patient experience depends on the ability to measure their associations with modifiable factors.10,11
Rationale
With these motivations, a new PREM with less ceiling effect and a greater distribution of scores was developed. Since the most useful items addressed trust in the clinician, the instrument was labeled the Trust and Experience with Clinicians Scale (TRECS). 13 In development, we noted that TRECS correlates well with a legacy experience measure, the Jefferson Scale of Patient's Perceptions of Physician Empathy (JSPPPE), while comparatively decreasing the ceiling effect. Having developed a PREM with greater variance and less ceiling effect, the next step is to determine whether we are able to detect associations with specific patient, care unit, and clinician factors and patient experience. This study addresses patient factors. Knowledge of patients’ personal factors associated with variation in patient experience has the potential to inform efforts to improve the experience of care.
Questions
Among a cohort of patients seeking musculoskeletal specialty care, we asked: (1) What patient factors are associated with patient experience measured using TRECS? (2) What patient factors are associated with patient experience measured using the legacy tool JSPPPE?
Methods
Study Design and Setting
In a cross-sectional study approved by the institutional review board (STUDY00004831), all new and returning adult (age 18-89 years) patients receiving musculoskeletal specialty care at one of 4 outpatient offices in an urban area were invited to participate. People with cognitive impairment or people whose primary language was not English were excluded. A research assistant not involved in patient care invited people to complete a survey on a Health Insurance and Portability Accounting Act compliant tablet (Research Electronic Data Capture [REDCap], Vanderbilt, TN) or personal electronic device. Verbal consent to participate and completion of the survey was considered informed consent.
Participants
We recruited 177 patients to participate. Four patients started the questionnaire but did not get past the initial demographics page. This is relatively common when people use their own device, which we relied on due to limited tablets at some research sites. One hundred seventy-three of 177 patients (98%) completed the entire questionnaire and were analyzed. The mean age of the participants was 54 ± 16 years. Fifty-one percent of participants identified as women (89 of 173) and 67% (116 of 173) identified as white (Table 1). More than half the participants sought care for an upper extremity concern (112 patients, 65%).
Demographics of Cohort.
Value is displayed as median with interquartile range for continuous variables with nonnormal distribution, and as a number with percentage for categorical variables.
Measured Explanatory Variables
We measured patient demographics, visit-related information, mental health (feelings of distress and unhelpful thoughts), and social health. Feelings of distress were measured using 3 items answered on a 5-point Likert scale (strongly disagree to strongly agree), where higher scores indicate more severe feelings of distress. 14 Unhelpful thoughts regarding symptoms were measured using 3 items using the same 5-point Likert scale, where higher scores indicate more severe unhelpful thinking. 14 Social health was measured using a 9-item patient-reported measure, including statements about their financial security (5 items) and social support status (4 items) rated on a 5-point Likert scale. 15 Higher scores indicate more favorable social circumstances. Measure of pain intensity was intentionally not measured because they are so strongly associated with measures of mental and social health that the collinearity distorts multivariable statistical models.
Primary Outcome Measures
We measured patient experience using the 7-item TRECS as the primary outcome and the JSPPPE as the secondary outcome. TRECS-7 is a quantitative measure of patient trust and experience in the clinician, in which statements are rated on a 5-point Likert scale. Examples of statements include “My provider seems concerned about me or my family,” or “I trust my provider would tell me if a mistake was made about my treatment.” Higher scores indicate a greater level of trust in the clinician. The JSPPPE is a measure of patient perception of clinician empathy. The patient rates the clinician on 5 statements using a 5-point Likert scale from strongly disagree to strongly agree. Examples of statements include: “Understands my feelings and concerns” or “Can view things from my perspective (see things as I see them).” 16
Statistical Analysis
We performed descriptive statistics for all participants (Table 1). Categorical variables were reported as percentages with frequency, and continuous variables were reported as mean with standard deviation (SD) or median with interquartile range (IQR), depending on data distribution for normally or nonnormally distributed data, respectively. We first used bivariate analyses to identify variables associated with TRECS and JSPPPE. We used Spearman rank correlation to assess continuous explanatory variables, Mann-Whitney U tests for dichotomous explanatory variables, and Kruskal-Wallis tests for categorical explanatory variables. We then constructed 3 different models with (1) all explanatory variables, (2) variables with P < .20 in bivariate, and (3) variables with P < .10 in bivariate, and assessed the Bayesian information criterion (BIC) value to determine the best fitting model (lowest BIC value: P < .10). We sought factors independently associated with TRECS and JSPPPE, accounting for potential confounders, using both multivariable linear regression as well as negative binomial regression models and only included the model with the lowest BIC value (the best model fit: multivariable linear regression). We reported regression coefficients, 95% confidence intervals (95% CI), P-value, and semipartial R2. We also assessed the overlap between TRECS and JSPPPE using Spearman's rank correlation with nonparametric bootstrapping (with replacement, n = 1000). Due to growing concerns that collinearity between mental and social health measures can distort the results of regressions, 14 we ran a posthoc k-means cluster analysis to identify statistical groupings of mental and social health factors in order to limit problems with collinearity. We allowed up to 20 clusters to be generated and identified the optimal number of clusters to retain using the “elbow” method described by Yuan and Yang while aiming to maintain at least 10 observations per cluster. 17 We decided to select both the 3-cluster and 6-cluster solutions, considering the “elbow” method pointed to the 6-cluster solution, whereas only the 3-cluster solution has at least 10 observations in each cluster. We believe both clusters are informative and therefore compared the mean differences in TRECS and JSPPPE between the identified clusters using one-way ANOVA tests for both cluster solutions. All variables with a P-value below .05 were considered statistically significant.
An a priori sample size calculation determined that 135 patients were needed for 80% statistical power to determine whether unhelpful thoughts and feelings of distress account for 10% or more of the variation in TRECS in a linear regression with 10 explanatory variables, if the complete model explains 25% or more of the overall variability. We enrolled 177 patients as we wanted to power the study to also compare 2 different methods for including variables in multivariable regression analysis: (1) including all available confounders, (2) selecting variables based on P value in bivariate analysis (P < .10 or P < .20).
Results
What Factors Are Associated With Patient Experience Measured Using TRECS?
Accounting for potential confounding factors identified in bivariate analysis (such as new or return visit, specific care unit, and patient self-reported race/ethnicity), no factors were associated with TRECS (Table 2). An additional posthoc cluster analysis was performed that identified both 3 and 6 cluster models of statistical groupings of levels of thoughts, emotions, and social health. In the 3-cluster solution, statistical groupings with healthier mindset and better social health were associated with modestly greater trust and experience with the physician (Table 3). The 6-cluster solution identified more complex variation in TRECS based on variation in both thoughts and emotions, as well as social health (Table 4). The highest level of trust and experience in the clinician was associated with very good social health and healthy thoughts and emotions (group 6). Both compromised social health and less healthy thoughts and emotions were associated with more notable decrements in levels of trust and experience than in the 3-cluster solution.
Multivariable Linear Regression Analysis of Factors Associated With the 7-Item Version of the Trust and Experience With Clinician Scale (TRECS).
Model fit statistics: R2 = 0.18, adjusted R2 = 0.12, AIC = 972.
Clusters Associated With Trust and Experience With the Clinician Scale (TRECS) With the 3-Cluster Solution.
P = .03, TRECS, Trust and Experience With Clinician Scale. ^ P = .09, JSPPPE, Jefferson's Scale of Patient Perceptions of Physician Empathy.
Clusters Associated With Trust and Experience With the Clinician Scale (TRECS) With 6-Cluster Solution.
P = .003, TRECS, Trust and Experience With Clinician Scale. ^ P = .009, JSPPPE, Jefferson's Scale of Patient Perceptions of Physician Empathy.
What Factors Are Associated With Patient Experience Measured Using JSPPPE?
Accounting for potential confounding factors such as new or return visit and patient self-reported race/ethnicity, we found that only self-described Hispanic/Latino ethnicity was associated with lower patient perceived physician empathy (RC = –2.8, 95% CI = –4.9 to −0.63; P = .012) (Table 5). The clusters identified in the 6-cluster, but not the 3-cluster solution, were associated with mean differences in perceived clinician empathy (JSPPE; Tables 3 and 4).
Multivariable Linear Regression Analysis of Factors Associated With JSPPPE.
P-values < 0.05 are bolded. Model fit statistics: R2 = 0.13, adjusted R2 = 0.091, AIC = 971.
Discussion
Current PREMs are limited by notable ceiling effects and a non-Gaussian distribution. 4 Such clustered responses at the top end of the scale limit the ability to identify modifiable factors that can help clinicians and care units to learn and improve their patients’ experience. PREMs correlate with each other, but, to date, they have modest and inconsistent correlations with other factors, such as patient personal factors. In this experiment, we tested whether TRECS could identify patient factors associated with better patient experience. In multivariable analysis, which is susceptible to multicollinearity and may be less well suited to identifying complex and nonlinear relationships, we found that TRECS was not associated with any factors, while JSPPPE was only associated with self-described Hispanic/Latino ethnicity. In a posthoc analysis using cluster analysis, a statistical technique that better accounts for collinearity and complex relationships, we selected 2 models with good performance that were both instructive. The 3-group model identified a relatively simple relationship between a better mindset (thoughts and emotions) and circumstances (social health), and modestly higher levels of trust and experience with the clinician. The 6-group solution identified some greater complexity, with variations in groupings of levels of less healthy thoughts and feelings and lower social health. The associations of experience measures (TRECS and JSPPE) noted with the cluster analysis are of interest because they indicate that the limited relationships in multivariable models may relate to relative collinearity between personal factors. The ability of an experience measure with a lower ceiling effect combined with cluster analysis to account for collinearity of patient personal factors has the potential to help individual clinicians and their organizations strategize and interpret their patient's experience as, in part, reflecting and representing aspects of their overall health. Considered in this regard, efforts to improve patient experience of care can anticipate the importance of social disadvantage and less healthy mindsets to a positive patient experience and develop strategies to help patients experiencing these psychosocial aspects of illness navigate care.
What Factors Are Associated With Patient Experience Ratings Measured Using TRECS and JSPPE?
The finding in the linear regression that no patient factors are associated with TRECS, and that JSPPPE was modestly associated with only Hispanic/Latino ethnicity, suggests that even a quantitative experience measure with less ceiling effects and more Gaussian distribution (TRECS) may not correlate with other measurable patient factors such as mental and social health factors. The finding of the planned analysis, that no matter a person's mindset and circumstances, it is possible to have a good experience, is consistent with prior research on PREMs by us and others.9,18,19 But the finding of associations of psychosocial factors with variation in levels of patient experience suggests that collinearity among the mental and social health variables might be distorting and limiting the linear regression models. The association with TRECS in the cluster analysis may demonstrate the advantage of a more Gaussian distribution with lower ceiling effect for helping identify important associations. This is relevant to the related difficulty in identifying clinician factors associated with PREM scores. 20 It is implausible that neither patient factors nor clinician communication effectiveness are related to levels of patient experience. More plausible is that we have not developed the measurements and analyses needed to identify important associations. Some constructs that merit additional study for associated with PREM scores include intolerance of uncertainty, inflexible thinking (cognitive fusion). Our speculation is that discordance between the clinician's evaluation of the sensations patient's feel and the patient's inner narrative around pain and/or injury may be a key threat to a trusting relationship. 9 Another consideration is the preliminary evidence that personality disorders, for instance, might be associated with worse patient-reported experience measures. 21 The cluster analysis shows that complex nonlinear statistical analyses that better account for the interrelationship of patient personal factors are able to discern factors associated with varied levels of experience. Using these techniques, we can see that patient factors are one contributor to diminished patient experience in the form of trust in the clinician and perceived clinician empathy. The same may be identified in a more sophisticated analysis of clinician factors.
There were several limitations to this study. First, we used specific, short-form measures of mental and social health. A study using longer measures, which have more variation and less potential for floor and ceiling effects, might have different relationships to TRECS. That said, in prior work, even a few items about mindset and circumstances have identified associations with levels of comfort and capability, so if there are important associations of psychosocial factors with PREM scores, we should be able to detect them with these items. Second, this study included all musculoskeletal conditions and noted the typical, relatively high levels of trust and perceived empathy. Studies of specific diagnoses or subpopulations of patients that are known or suspected to be associated with lower patient ratings of experience—circumstances where the patient's experience conflicts with the medical facts, such as misperception of new pain from a degenerative condition as an injury, for instance—might identify patient factors associated with lower ratings of experience.12,22 As another consideration, diffuse and disproportionate pains can be associated with greater feelings of distress and might be associated with worse patient experience, although some data suggests, on average, they are not.23-25 Another consideration is the use of items that address thoughts and feelings regarding bodily sensations rather than levels of anxiety and depression in general. This approach was taken to avoid the known floor effects and nonforthright completion of measures of general anxiety and depression that are known to occur in musculoskeletal specialty care, and there may or may not be a disadvantage to this approach.11,26 Lastly, the ability to choose the number of statistical clusters in cluster analysis may seem unsettlingly subjective. The reader should keep in mind that there is no best or correct statistical approach, that all of them have subjective aspects that must be informed by expertise, and that the collective analysis can increase knowledge and inform future experiments. Given that the cluster analysis was unplanned, it should be considered provisional and hypothesis-generating until reproduced in planned experiments. Another consideration is that others have not used and validated the TRECS. This second validation study by our group found results consistent with the development, but external validation is needed. All of the Hispanic patients included spoke English as their primary language, so language difficulties with the questionnaires are unlikely.
Conclusion
Although TRECS provides a more varied distribution with lower ceiling effects than other PREMs, no associations of patient factors with patient experience were identified in standard multivariable analysis. However, using cluster analysis—a statistical technique that can account for collinearity between mental and social health variables and complex nonlinear relationships—we found associations between statistical groupings of patient mindsets and circumstances that are associated with variations in both perceived clinician empathy (JSPPPE) and trust and experiences with the clinician (TRECS). This represents evidence that the combination of a more Gaussian measure of patient experience with a lower ceiling effect, and employing more suitable statistical analyses, may facilitate the identification of modifiable patient and clinician factors with the potential to improve patient experience.
Footnotes
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was not funded. The work was performed at the University of Texas at Austin, Austin, Texas, USA.
Ethical Approval
The IRB determined that this protocol was approved and exempted from further review under 45 CFR 46.104 (2)(ii) Tests, surveys, interviews, or observation (low risk). IRB ID: STUDY00004831.
Statement of Human and Animal Rights
Not applicable.
Informed Consent
Not applicable.
