Abstract
A health care provider’s ability to establish and maintain a strong therapeutic relationship epitomizes client-centered care (Constand et al., 2014; Zill et al., 2015) and occupational therapy practice (American Occupational Therapy Association, 2014). A provider’s approach to the therapeutic relationship is influenced by technical, interpersonal, and reflective competencies (Crepeau & Garren, 2011) and has been shown to influence clients’ therapeutic engagement (Bright et al., 2015). The Intentional Relationship Model (IRM; Taylor, 2020). is a widely used theoretical model for the therapeutic use of self in occupational therapy (Solman & Clouston, 2016). It promotes a reasoning process for understanding interpersonal behavior, anticipating inevitable interpersonal events that arise throughout the course of therapy, and exercising intentionality in communication. The IRM proposes that effective communication consists of six distinct communication modes: advocating, collaborating, empathizing, encouraging, instructing, and problem solving. Advocating involves awareness raising and connecting clients with people and resources in the community. Collaborating requires the provider to shift the power of decision making to the client. Empathizing involves striving to understand the client’s thoughts, feelings, and behaviors through validation and gentle inquiry. Encouraging involves instilling hope and providing positive reinforcement. Instructing characterizes a structured approach to teaching and providing feedback, and problem solving involves guiding the client’s thinking by means of Socratic questioning and outlining options.
The Clinical Assessment of Modes (CAM; Taylor & Popova, 2019) was developed on the basis of the IRM. The CAM offers a systematic approach to evaluating a provider’s communication overall and according to the six IRM modes. Four different versions of the CAM have demonstrated adequate construct validity for measuring individual modes of communication in inpatient and outpatient rehabilitation: provider self-report, observer report, client self-report of preferred communication, and client self-report of experienced communication (Fan & Taylor, 2016). The primary aim of the current study was to replicate Fan and Taylor’s (2016) findings for the provider self-report version of the CAM, the CAM–Therapist version (CAM–T; Taylor & Popova, 2019), in acute care and acute inpatient rehabilitation settings with an interdisciplinary team of providers. The secondary aim was to expand on Fan and Taylor’s findings by examining the internal consistency and structural validity of the CAM–T as a measure of overall communication and of communication within the six individual modes.
Method
The institutional review board at the University of Illinois at Chicago approved this study, which was conducted using psychometric analyses with cross-sectional sampling. Participants were recruited from acute care and acute inpatient rehabilitation units of a hospital and screened for eligibility. Written informed consent was collected in English. Participants had to be at least age 18 yr to enroll. Client participants were eligible to participate on the basis of being determined to be medically stable to take part in therapy and having a referral for occupational or physical therapy evaluation from the attending physician. Provider participants were recruited on the basis of working full or part time in occupational therapy, physical therapy, or nursing. Providers completed the CAM–T immediately after each session.
Measures
The CAM–T consists of 30 items (Table 1 contains a list of abbreviated item descriptions). The items are separated into six subscales (i.e., Advocating, Collaborating, Empathizing, Encouraging, Instructing, and Problem-Solving), each with five items reflecting one of the six IRM modes. Each item is rated on a 5-point scale (0 = never, 1 = rarely, 2 = occasionally, 3 = frequently, 4 = very frequently). The CAM–T is scored in two ways: (1) An average score for overall communication is obtained by averaging responses to all 30 items and (2) an average score for each of the six modes is obtained by averaging responses to the five items of each subscale.
Item Fit Statistics
Note. IM = item measure; MnSq = mean square; SE = standard error; Zstd = Z standard.
Misfit with the expectations of the Rasch model (MnSq ≥ 1.4 and Zstd ≥ 2.0).
Analysis
Classical Test Theory Approach
We evaluated internal consistency to determine whether the items on the CAM–T worked together to measure the same construct of communication, overall and within each of the six subscales. Reliability data were analyzed using IBM SPSS Statistics for Windows (Version 22.0; IBM Corp., Armonk, NY). Internal consistency was evaluated using Cronbach’s α, where <.59 = poor, .60–.69 = questionable, .70–.79 = acceptable, .80–.89 = good, and ≥.90 = excellent (Tavakol & Dennick, 2011).
Rasch Analytic Approach
Rasch analysis was performed using the Winsteps® Rasch measurement computer program (Version 3.93.0; Winsteps.com, Beaverton, OR). Rasch analysis evaluates the psychometric properties of an assessment by comparing item and person responses against a Rasch-generated model, thus accounting for the difficulty of the items on the assessment and the ability of the people completing it. The Rasch approach makes it possible to evaluate the structural validity of an assessment by generating a latent trait model. Each Rasch model is used to confirm whether the items on the assessment measure an underlying trait or concept that accounts for a person’s response pattern on the assessment. We used the Andrich Rating Scale Model (Bond & Fox, 2015) approach to evaluate five aspects of psychometric functioning: (1) rating scale functioning, (2) dimensionality, (3) item and person fit, (4) item targeting, and (5) item and person separation.
As evidence of appropriate rating scale functioning, we expected that each rating category would have ≥10 responses; an outfit mean square (MnSq) ≤2.0; and evidence of rating scale effectiveness through monotonic advancement (static increases) in observed averages, with more frequent rating categories demonstrating greater measure scores (Linacre, 2002). Dimensionality was evaluated to determine whether the overall instrument and the individual subscales were measuring a single construct of communication. We expected that ≥50% of variance would be explained by the measure (Smith & Miao, 1994).
Individual ratings were examined according to item and person fit to the expectations of the Rasch-generated model. The infit statistics were expected to be 0.6 ≤ MnSq ≤ 1.4 and –2.0 ≤ Z standard (Zstd) ≤ 2.0 (Bond & Fox, 2015). The infit statistics above the cutoff were suspected to be erratic, demonstrating item misfit and threat to structural validity. The infit statistics below the cutoff were suspected to be highly predictable, demonstrating item overfit and the possibility of item redundancy. Keeping in mind that item overfit does not pose a significant threat to structural validity (Bond & Fox, 2015), we examined only item and person misfit. For overall communication, we expected that ≥95% of the items on the CAM–T would demonstrate appropriate fit. For the individual mode subscales, the item fit criteria were set to ≥80% of items to accommodate the small number of items on each subscale. The person fit criteria were set to ≥90% for overall communication and individual mode subscales.
We examined ceiling effects, floor effects, and person–item hierarchy maps for item targeting (the match between item difficulty and participant ability). Ceiling and floor effects were suspected if >15% of the participants received the maximum or minimum score (McHorney & Tarlov, 1995). We evaluated item hierarchies according to item measure scores and standard errors. Item separation was examined for appropriateness of the sample size for evaluating item difficulty hierarchy. Person separation was examined for assessment sensitivity to the performance levels of the providers in the sample. Separation coefficients ≥2.00 with reliabilities ≥.80 were expected (Arnadóttir & Fisher, 2008).
Results
Participants
A convenience sample of 32 providers and 96 clients (3 clients per enrolled provider) from one hospital agreed to participate. Providers’ self-reports from 96 first-time provider–client interactions were collected over 14 mo. Of the 96 interactions, the majority (n = 66) took place in an acute care setting. The interactions lasted between 4 and 61 min (mean [M] = 22.6, standard deviation [SD] = 12.3). The providers included occupational therapists (n = 8), an occupational therapy assistant (n = 1), physical therapists (n = 6), nurses (n = 15), and nursing assistants (n = 2). The average age ranged from 22 to 54 yr (M = 33.7, SD = 8.3) for the providers and from 25 to 88 yr (M = 55.3, SD = 12.7) for the clients. Most providers (n = 25) and clients (n = 52) identified as female. The providers identified as White or Caucasian (n = 16), Asian (n = 11), Black or African-American (n = 3), and other (n = 2). The clients identified as Black or African-American (n = 58), White or Caucasian (n = 28), Hispanic or Latino (n = 9), and other (n = 1).
Classical Test Theory Approach
There was one missing response for the CAM–T, which was omitted from the analysis. The Cronbach’s α statistics met the criteria for overall communication (α = .93) and the six subscales: Advocating (α = .89), Collaborating (α = .75), Empathizing (α = .62), Encouraging (α = .81), Instructing (α = .69), and Problem-Solving (α = .84).
Rasch Analytic Approach
The CAM–T demonstrated good rating scale functioning, meeting all predefined criteria for overall communication and the individual subscales. Further examination of the response distribution suggested that the providers were least likely to endorse the “Frequently” category for the Advocating subscale and the “Never” and “Rarely” categories for all other subscales. Overall communication and all individual subscales met the criteria for dimensionality and item fit (Tables 1 and 2). Person fit fell below the criteria for overall communication and for the Collaborating, Empathizing, and Encouraging subscales (Table 2). The Advocating subscale demonstrated a greater than expected floor effect, whereas the Encouraging subscale demonstrated a greater than expected ceiling effect (Table 2). Visual inspection of item–person hierarchy maps further supported these findings; the providers reported a low frequency of communication on the Advocating items and a high frequency of communication on the Encouraging items. Overall communication and four individual subscales met the criteria for item separation; the Encouraging and Problem-Solving subscales did not (Table 2). Person separation criteria were met for overall communication but not for the individual subscales (Table 2).
Summary of Rasch Analysis Findings
Did not meet the cutoff criteria.
Discussion
The IRM contends that a provider’s communication is most likely to be therapeutic when the provider is interacting within one of its six communication modes in a manner that matches the client’s interpersonal needs. The CAM–T was designed to support providers’ capacity to evaluate and self-reflect on their use of therapeutic communication overall and specific to the six modes defined by the IRM. This study builds on findings published by Fan and Taylor (2016) by offering evidence for the reliability and validity of the CAM–T in acute care and acute inpatient rehabilitation settings with an interdisciplinary team of health care providers. In addition, this study supports the reliability and validity of the CAM–T as a measure of overall communication and communication within the six IRM modes.
The CAM–T demonstrated good internal consistency and structural validity for evaluating providers’ overall communication. The structural validity of the CAM–T for evaluating overall communication was supported through evidence of good rating scale functioning, unidimensionality, item fit, item targeting, and item and person separation. Although person fit was lower than expected, the statistic was 4% below the cutoff and was not deemed to pose a significant threat to construct validity.
The six subscales demonstrated adequate internal consistency and structural validity for evaluating individual mode use. Each subscale demonstrated appropriate rating scale functioning and measured a unidimensional construct of communication; however, a number of potential threats were noted. The providers were least likely to endorse the “frequently” category for the Advocating subscale and the “never” and rarely” categories for all other individual subscales. These findings are consistent with those reported by Fan and Taylor (2016), who noted that the “never” category was least used by providers in inpatient and outpatient rehabilitation settings. Further analysis and potential revision of the rating scale are necessary.
The Collaborating, Empathizing, and Encouraging subscales showed lower than expected person fit. Because the person fit was close to the cutoff, these findings were not deemed to pose a significant enough threat to structural validity to justify item revision at this time. The Advocating subscale demonstrated a tendency toward a floor effect, and the providers reported infrequent use of this mode in their sessions. The Encouraging subscale demonstrated a tendency toward a ceiling effect, and providers reported frequently using this mode in their sessions. According to the IRM, there is no specified item or mode hierarchy; however, it is possible that providers may demonstrate preferential mode use that is geared toward the unique needs of the population served in the practice setting.
The evidence that the six CAM–T subscales could be used to separate providers by their ability to use different modes of communication was insufficient. Low item separation for the Encouraging and Problem-Solving subscales also raised concerns regarding the reliability of these subscales in distinguishing among different levels of item difficulty. Given that Fan and Taylor (2016) reported appropriate item fit, item targeting, and item separation across the six subscales, the present findings may be a result of sampling differences. Provided that the goal of effective communication is to adapt mode use to the client’s unique needs, the IRM argues that no single mode should be superior to another within a provider’s conceptual and practical approach. Given this theoretical underpinning, low item or person separation is not as concerning as it might be when evaluating a scale that was grounded in a theoretical expectation involving hierarchical differentiation between items and persons, such as one that measured occupational functioning, performance, or participation.
The present findings are strengthened by the interdisciplinary nature of the provider sample. Although it is unclear how this may have influenced the validity and reliability of individual mode subscale findings, the results support the use of the CAM–T with providers outside the field of occupational therapy. Communication has been shown to be an important contributor to client outcomes across health care disciplines. Among others, occupational therapists and occupational therapy assistants can use the CAM–T to address interpersonal and communication concerns within an interdisciplinary rehabilitation team.
Limitations
The conclusions about the psychometric properties of the CAM–T must be interpreted with the following limitations in mind. The sample size, although appropriate for the analyses conducted in this study, was not ideal for Rasch. Clients who elected to participate in this study may have been more agreeable or interpersonally comfortable while interacting with providers. This would make it easier for providers to remain therapeutic in their communication. In addition, convenience sampling may have inadvertently attracted providers who felt comfortable being evaluated with respect to their communication. Moreover, the use of a self-report assessment is inevitably influenced by the social desirability response bias; as a result, the reported communication may not be an accurate indication of what actually occurred, or typically occurs, during certain client–provider interactions. Future research examining the comparison of therapist and observer perspectives is warranted.
Recommendations for Future Research
The findings from this study, and those reported by Fan and Taylor (2016), suggest that revisions may be necessary to improve the functioning of the 5-point rating scale selected for the CAM–T. Moreover, the reliability and validity of the CAM–T should be further evaluated with different client populations and in a wider range of practice settings.
Implications for Occupational Therapy Practice
This study has two implications for occupational therapy practice:
The CAM–T is a reliable and valid measure for use in acute care and acute rehabilitation settings.
The CAM–T evaluates overall therapeutic communication and individual mode use as described by the IRM.
Conclusion
This study supports the CAM–T’s reliability and validity as a self-report measure of providers’ communication with clients in acute care and acute rehabilitation settings. Providers’ interpersonal approach to care can affect client engagement and, ultimately, client outcomes. Integration of psychometrically sound tools that can be used to guide critical reflection related to provider–client interaction can strengthen clinical practice and outcomes.
Footnotes
Acknowledgments
We thank our clinical partners Kay McGee and Melissa Lara for assisting us with participant recruitment. We also thank our dedicated research assistants: Jenna Colangelo, Kaitlin Ibara, Rikki Ostrowski, Ariana Rodriguez, and Jennifer Wescott.