Abstract
Keywords
Clients’ perceptions of therapy have been viewed as abstract and multidimensional phenomena (Hudak & Wright, 2000). Clients with positive perceptions are more likely to adhere to treatment plans and continue to seek therapy when needed (Mitchell & Selmes, 2007). According to Palmadottir (2003), the client–therapist relationship is one aspect that influences clients’ perceptions of therapy outcomes. However, a study of practicing occupational therapists found that an undeniable high proportion of clients (between 2% and 30%, depending upon the behavior) experience high levels of emotional distress and have difficulties relating during therapy (Taylor, Lee, Kielhofner, & Ketkar, 2009). Moreover, the research on specific interpersonal strategies used in response to clients’ needs and preferences is limited (Taylor, Lee, & Kielhofner, 2011). Because of the complexity inherent in this relationship, a valid and reliable measure to evaluate therapeutic communication is needed in rehabilitation (Vegni, Mauri, D’Apice, & Moja, 2010) and, specifically, in occupational therapy.
Intentional Relationship Model
As a way to integrate and conceptualize therapeutic communication in rehabilitation, Taylor (2008) introduced the Intentional Relationship Model (IRM) within the field of occupational therapy. The IRM defines the therapeutic relationship in terms of three major components that are woven together in the interpersonal reasoning process: (1) the client’s interpersonal characteristics and expectations about relating, (2) the therapist’s choice of communication modes, and (3) the events that occur during therapy that inevitably affect the process and outcomes of the relationship. The IRM specifies that six communication modes form the foundation of a therapist’s interpersonal approach: advocating, collaborating, empathizing, encouraging, instructing, and problem solving. For this reason, we felt it was important to develop assessments that would measure these modes.
The advocating mode involves statements that normalize a client’s experience and raise the client’s awareness about his or her legal rights to equal opportunity and other resources for people with disabilities. The collaborating mode emphasizes the client’s autonomy in setting therapy goals and choice in directing the course of his or her therapy experience. Using the empathizing mode, therapists strive to understand the client’s experience through gentle inquiry, summarizing statements, validation of negativity, and mirroring. The encouraging mode comprises positive reinforcement, confidence building, hope-building statements, and the use of humor, as appropriate. Using the instructing mode, therapists educate the client and structure the treatment process so that the client knows what is expected. In addition, therapists demonstrate how to perform a task, explain the rationale for doing something, set boundaries and limits on the client, and provide corrective feedback, when appropriate. In the problem-solving mode, therapists ask strategic questions that serve the agenda of increasing the client’s ability to reason through problems. This mode helps the client clarify his or her thoughts, identify alternatives, and weigh the pros and cons of the options.
A retrospective survey study of more than 700 practicing therapists (Taylor et al., 2011) found that therapists frequently endorsed items that were consistent with each of the six IRM communication modes when working in interpersonally challenging situations. Specifically, therapists who experienced more challenging situations reported higher use of all six modes and a preference for using the instructing and problem-solving modes (Taylor et al., 2011). Although a preliminary confirmation of mode use in practice was an important step in operationalizing therapeutic mode use, the study was limited because it relied on a retroactive self-report questionnaire to collect the data. Data were not collected from the client’s perspective, nor from the perspective of an independent observer of the client–therapist interaction. Moreover, the methodological approach did not allow for the assessment of mode use during real-time clinical situations. Outside of these studies, there have been no other investigations of these issues in the field to date.
By contrast, the Clinical Assessment of Modes (CAM) allows for data collection from multiple perspectives and offers an observational version designed for use during real-time practice interactions.
Clinical Assessment of Modes
The CAM is an applied practice assessment derived from central concepts of the IRM (Taylor, 2008). Each of its six subscales contains items that define one of the six IRM modes (advocating, collaborating, empathizing, encouraging, instructing, and problem solving). Four nearly identical versions of the questionnaire exist, with each version capturing a different vantage point of the therapist’s use of the six IRM communication modes. The four versions are as follows: the patient preference version (CAM–P; Taylor & Fan, 2015b), which measures the client’s preference for therapist mode use going into the therapy relationship; the patient experiences version (CAM–E), which measures the client’s actual experience of the therapist’s mode use (Taylor & Fan, 2015a), the therapist self-report version (CAM–T; Taylor et al., 2013), which measures the therapist’s self-reported perspective; and an observational rating version (CAM–O) that allows an independent rater to rate the therapist’s mode use during the session (Fan, Taylor, Wong, & Zubel, 2013).
Study Objectives
To adequately evaluate the properties of the CAM subscales (which were designed to measure the central constructs of IRM modes), we used Rasch analysis, which allows for the testing of the reliability (i.e., internal consistency) and validity (i.e., rating scale analysis, dimensionality analysis, test-targeting analysis, and item separation analysis) of the four versions of the CAM. For each version, four specific objectives were explored: (1) initial rating scale evaluation, (2) dimensionality evaluation, (3) targeting evaluation, and (4) item separation analyses. The first objective was to assess whether the range of response options was appropriate for the respondents (i.e., whether a five-category ordinal scale represented the appropriate number of options). The second was to measure the unidimensionality of the assessment (i.e., whether each mode subscale measured a single mode or approach to therapeutic communication). The third was to measure targeting appropriateness (i.e., whether the five items comprising each mode subscale effectively defined a range of preferences, experiences, and ratings within that mode, from low to high). The final objective was to measure item separation reliability (i.e., the internal consistency reliability; Linacre, 2011).
Method
Participants
This study used two types of participants: clients undergoing inpatient and outpatient rehabilitation and their treating therapists. In addition, 7 research assistants who underwent one-on-one training and supervision in rating the therapists’ use of interpersonal modes served as raters for the CAM–O.
Participant Group 1: Clients.
Initially, 120 clients receiving rehabilitation services at the University of Illinois Hospital and Health Sciences System consented to participate in this study. The CAM–P was administered to 63 enrolled inpatients at intake. The CAM–E was administered to the same group of 63 plus an additional 47 outpatients immediately after the third treatment session, for a total CAM–E sample of 110. Nine of the 120 clients who originally consented to participate were transferred out before the third session, and 1 refused to complete the CAM–E.
Participant Group 2: Therapists.
Thirty-eight therapists employed at the hospital during the study period were invited to participate in this study. Participation involved therapists completing the CAM–T at the end of the third session and allowing themselves to be observed during the first and third treatment sessions.
Measures
Each version of the CAM contains 30 items, with each of the six subscales (i.e., Advocating, Collaborating, Empathizing, Encouraging, Instructing, and Problem Solving) containing 5 items. Table 1 contains the items that were included on the CAM–O subscale. Each version contains the same items; the only difference between the versions involves the timing of administration (i.e., before, during, or after therapy) and the reporting perspective (client self-report, therapist self-report, and observational), with the voice changing from the first to the third person in the presentation of items.
CAM–O Subscale Items
Note. CAM–O = Clinical Assessment of Modes, observer rating scale. Each version of the CAM contains the same items; the versions differ only in timing of administration and reporting perspective.
In this study, all scales were scored on a 5-point ordinal scale ranging from 1 (not at all important [CAM–P] or never [CAM–E, CAM–T, CAM–O]) to 5 (extremely important [CAM–P] or very frequently [CAM–E, CAM–T, CAM–O]). A higher score for a given mode subscale indicated greater endorsement of the specific approach to communication (i.e., therapeutic mode) being measured.
Procedures
The study was approved by the human subjects division of the University of Illinois at Chicago’s internal review board, and all participants underwent informed consent procedures. Therapist participants were assigned to client participants on the basis of diagnosis as it matched each therapist’s area of practice and expertise. The CAM–P was administered only to the inpatient participants during a routine intake session. The CAM–E was administered to the inpatient and outpatient participants immediately after the third treatment session. The 7 observer-raters were assigned to observe the treatment sessions and administered the CAM–O during the course of the first and the third treatment sessions on the basis of scheduling availability. The CAM–T was administered to therapists after the third treatment session for each of the enrolled participants.
Data Analysis
Using the Rasch model (Wright, 1999), 24 separate analyses were completed, 1 for each of the six modes for each version of the CAM. The idea of mode use is that there is no hierarchy but instead a range in the client’s personal preference for therapist mode use (in the case of the CAM–P) and the client's actual experience of the therapist using that mode (in the case of the CAM–E). Rasch techniques demonstrate the range of a client’s perception of therapeutic communication preference and experience and an order of likelihood of endorsement along a continuum.
In this study, descriptive statistics were analyzed using PASW Statistics (Version 19.0 for Windows; PASW Inc., Chicago), and the Rasch analysis was conducted using Facets (Version 3.86.1; Winsteps.com, Chicago; Linacre, 2011). The Rasch analyses were conducted in four stages: (1) initial rating scale evaluation, (2) dimensionality evaluation, (3) targeting evaluation, and (4) item separation analyses.
Rating Scale Evaluation
The following criteria (Linacre, 2002) were adopted for the examination of each version of the CAM rating scale:
Each rating category should have more than 10 clients.
Average measures of each rating category should advance monotonically.
The outfit mean square (MnSq) fit statistic should be less than 2.
Four-step calibrations should increase within the five CAM rating categories.
Rating category disordering was examined by diagnosing the items’ average measures at this stage. It is assumed that clients who preferred or perceived lower levels of communication within a given mode were rated lower than clients who preferred or perceived higher levels of communication within that mode. In addition, to investigate whether the CAM’s 5-point Likert scale was appropriately interpreted and used by clients, therapists, and observers, the category probability curves were inspected.
Dimensionality Evaluation
A dimensionality evaluation was used to examine whether the items on each of the six subscales could be viewed independently—each as a single dimension representing each of the six communication modes. The dimensionality of the CAM was evaluated using goodness-of-fit statistics. The criteria were a MnSq fit statistic set at 0.6 to 1.4 with a standardized mean square (Zstd) of −2 to 2 to represent the item’s validity represented within the six latent traits (Bond & Fox, 2007).
Targeting Evaluation
The targeting evaluation was conducted by comparing mean item difficulty and mean person ability. The test items’ average level of endorsement was expected to match the targeted clients’ preference as closely as possible. Researchers have suggested the criterion of 0.5 logits as the acceptable difference (Lai & Eton, 2002). Ceiling and floor effects were examined at this stage to see whether more than 15% of the clients achieved the maximum and minimum scores, respectively (McHorney & Tarlov, 1995).
Item Separation Analysis
Item separation reliability was used to examine internal consistency (Linacre, 2011). A separation reliability value of .80 or greater (Arnadóttir & Fisher, 2008) was adopted to indicate that the CAM questionnaires had satisfactory internal consistency.
Results
Participant Demographics
Client participants ranged in age from 18 to 89 yr (mean = 50.14 yr, standard deviation = 15.46). Most were diagnosed with stroke or fracture, and there was little difference in the number of each gender. Most of the therapists (90%) identified the inpatient center as their primary work area, and 70% were female. (See Table 2 for detailed demographics of the client, therapist, and rater groups.)
Characteristics of Participants and Raters
Note. Percentages may total more than 100 because of rounding. — = not applicable; M = mean, SD = standard deviation.
Clinical Assessment of Modes, Client Preference Version
The CAM–P was administered to only the inpatient sample on intake. Therefore, the following results were based on an analysis of 63 participants.
Rating Scale Evaluation.
Among the 30 CAM–P items, 9 were endorsed at a rate of fewer than 10 clients per rating category. The least used rating category was not at all important. The outfit MnSq was <2.0. One-third of the items were found to have disordered step calibrations. The disordering in rating was confirmed by rating category probability curves and occurred for five of the six subscales (the Collaborating subscale was the exception). On the basis of these findings, the adjacent disordered categories were collapsed, and a reanalysis was conducted. After collapsing the adjacent disordered categories, the category disordering decreased to 3 of 30 items (10%). The problem of not having at least 10 observations per rating category was also addressed when the two least endorsed categories were collapsed. Originally, 9 items had fewer than 10 observations in Rating Category 1. After Rating Categories 1 and 2 were collapsed, all of the rating categories contained at least 10 observations.
Dimensionality Evaluation.
Table 3 shows the results of the Rasch analysis for each CAM–P subscale. Overall, 28 of the 30 CAM–P items (93.3%) fit the model’s expectation, except for Item 8 on the Instructing subscale and Item 18 on the Advocating subscale (Table 3).
Fit Statistics of the Four Versions of the CAM by Mode
Note. CAM = Clinical Assessment of Modes; CAM–E = client’s treatment experience; CAM–O = observer rating scale; CAM–P = client’s pretreatment preference; CAM–T = therapist’s self-reported perspective; MnSq = mean square; Zstd = standardized mean square.
Item did not meet the expectations of the Rasch model.
Targeting Evaluation.
The difference (1.27 logits) between the mean logit calibration of item and person exceeded the acceptable criterion of 0.5 logits, with the Instructing and Problem Solving subscales showing a tendency toward a ceiling effect (26.8% and 18.3% of clients, respectively, reached the maximum score). No floor effects were found (0% to 1.4%).
Item Separation Analysis.
Item separation reliabilities were found to be between .80 and .94 for each mode (Table 4), indicating that the CAM–P items on each mode subscale had acceptable internal consistency and also defined the construct of clients’ preference for different therapeutic communication modes.
Separation Reliability of the Subscales for Each Version of the Clinical Assessment of Modes
Note. Item separation reliability in Rasch is equivalent to the Cronbach’s α test reliability statistic. CAM = Clinical Assessment of Modes; CAM–E = client’s treatment experience; CAM–O = observer rating scale; CAM–P = client’s pretreatment preference; CAM–T = therapist’s self-reported perspective.
Separation reliability ≥ .80 (Arnadóttir & Fisher, 2008).
Clinical Assessment of Modes, Client Experience Version
Results from the CAM–E were based on an analysis of 110 enrolled inpatient and outpatient participants. To test for unanticipated differences between the inpatient and outpatient samples, χ2 and independent-samples t tests were used to examine demographic characteristics between the two groups. Because no significant differences were found, data from the two groups were combined for the following analyses.
Rating Scale Evaluation.
Of the 30 CAM–E items, 8 were endorsed at a rate of fewer than 10 clients per rating category. The least used rating categories were never and rarely. The outfit MnSq was <2.0. Fourteen of the 30 items (46.67%) were found to have disordered step calibrations when five rating categories were used. Rating category disordering was found across all six mode subscales. In addition, rating category probability curves demonstrated disordered thresholds for all six mode subscales. After collapsing adjacent disordered categories, the category disordering decreased to 3 of 30 items (10%). Only 4 items still had fewer than 10 observations.
Dimensionality Evaluation.
All five items (100%) on each CAM–E subscale fit the Rasch model (Table 3).
Targeting Evaluation.
The difference between mean item difficulty and mean person ability was 1.27 logits; items on the Collaborating, Empathizing, Instructing, and Problem Solving mode subscales showed a tendency toward a ceiling effect with 16.4%, 23.6%, 33.6%, and 24.5% of clients, respectively. No floor effects were found (0%–5.5%).
Item Separation Analysis.
The item separation reliabilities for the CAM–E ranged from .70 to .99 (Table 4).
Clinical Assessment of Modes, Therapist Self-Report Version
The CAM–T was administered to the 38 participating therapists, each of whom was paired with one or more client participants. A total of 110 observations were recorded for the Rasch analysis.
Rating Scale Evaluation.
Among the 30 CAM–T items, 14 were endorsed at a rate of fewer than 10 clients per rating category. The least used category in the CAM–T was never. The outfit MnSq was <2.0. Five items were found to have disordered step calibrations. Two of the six mode subscales showed disordered thresholds (Encouraging and Instructing). For all other mode subscales, the probability curves clearly demonstrated that item thresholds were properly ordered.
Dimensionality Evaluation.
All five items (100%) on each CAM–T subscale fit the model’s expectation, with acceptable MnSq and Zstd values (Table 3).
Targeting Evaluation.
The difference (1.13 logits) between mean logit calibration of item and person exceeded the acceptable criterion of 0.5 logits. No ceiling (0% to 8.0%) or floor effects were found (0% to 0.9%).
Item Separation Analysis.
The item separation reliabilities for the CAM–T ranged from .89 to .99 (Table 4).
Clinical Assessment of Modes, Observer Rating Version
At least two independent trained raters assessed a group of 59 clients derived from the larger group of 110 clients either at intake or during the third treatment session. A total of 196 observations were recorded for the Rasch analysis.
Rating Scale Evaluation.
Among the 30 CAM–O items, 9 were endorsed at a rate of fewer than 10 clients per rating category. The least used category on the CAM–O was never. The outfit MnSq was <2.0. Six items were found to have disordered step calibrations. The Advocating mode subscale showed disordered thresholds. For all other mode subscales, the probability curves clearly demonstrated that item thresholds were properly ordered.
Dimensionality Evaluation.
All five items (100%) on each CAM–O subscale fit the model’s expectation (Table 3).
Targeting Evaluation.
The difference (0.75 logits) between mean logit calibration of item and person exceeded the acceptable criterion of 0.5 logits. Items on the Instructing mode subscale showed a tendency toward a ceiling effect, with 16.3% of clients who reached maximum scores. No floor effects were found (0% to 2.6%).
Item Separation Analysis.
The item separation reliabilities for the CAM–O ranged from .75 to .99 (Table 4).
Discussion
This study evaluated the psychometric properties of the four versions of the CAM, which was developed for clinical rehabilitation practice and based on the IRM (Taylor, 2008). CAM items on the six mode subscales were verified to evaluate six different approaches to therapeutic communication: advocating, collaborating, empathizing, encouraging, instructing, and problem solving. Findings suggested that the CAM–P, CAM–E, CAM–T, and CAM–O are reliable and valid assessments.
Most of the items related well and constituted a unidimensional construct of therapeutic communication within each mode subscale and on each version of the CAM. Results confirmed that the six mode subscales represent distinct approaches to therapeutic communication. Across all four versions of the assessment, each subscale showed excellent internal consistency (.70–.99). Two items on the CAM–P misfit: Item 8 (“I want my therapist to tell me how to improve my performance or behavior”) and Item 18 (“I want my therapist to say things that help me to feel normal and like other people”). These results may need further study to verify whether the two items should be revised or deleted.
When items fit the model’s expectation, each of the ratings within the rating categories is expected to systematically take a turn showing the highest probability of endorsement (Pallant & Tennant, 2007). According to Pallant, Misajon, Bennett, and Manderson (2006), disordered categories happen when clients and therapists “have difficulty consistently discriminating between response options.” Findings on the client versions of the CAM (CAM–P and CAM–E) suggested that participants did not use the rating scale options in a consistent manner. This finding may be the result of the presence of too many response options or the options being interpreted differently between participants. Thus, rating categories in which the disordered thresholds occurred were collapsed, leading to a revision of these versions of the CAM scales (Taylor & Fan, 2015a, 2015b) so that they contained 4-point rating scales.
Additional findings revealed that clients rarely endorsed Rating Category 1 (not important at all) on the CAM–P, suggesting that all 30 items were viewed as important to their rehabilitation (to varying degrees). The results were consistent with findings from the targeting evaluation because items showed a tendency toward a ceiling effect, which indicated that the clients had higher expectations of all of the different approaches to therapeutic communication than the items and rating categories provided. This finding makes sense in light of the IRM (Taylor, 2008), which contends that all six therapeutic communication modes are therapeutic, depending on the client’s perspective and experience. Preferences tend to differ according to each client’s unique experiences as the therapy process unfolds. Accordingly, none of the versions of the assessment had a floor effect; clients perceived all items as desirable.
Limitations
This study had several limitations. First, participation was voluntary. Therefore, there is a chance that clients and therapists who were willing to participate were those who had more positive expectations of the therapeutic relationship or who had positive experiences with therapeutic relationships in the past. In addition, the therapist sample was small. Because of these sampling limitations, the findings may not generalize to other populations.
Implications for Occupational Therapy Practice
The purpose of this study is to examine the psychometric properties of a set of newly developed assessments based on the IRM. The assessments were designed to quantify aspects of therapeutic communication during interactions in rehabilitation. The main implications from this study are as follows:
The IRM (Taylor, 2008) provides a structured guide for understanding occupational therapy clients from an interpersonal perspective. Moreover, it articulates a reasoning process that provides practitioners with a means of selecting and applying six modes to anticipate and respond to challenging or emotional situations that inevitably arise during the therapy process. These modes are oriented toward advocating, collaborating, empathizing, encouraging, instructing, and problem solving.
This study used the Rasch model to analyze the psychometric properties of a four-version communication assessment, the CAM, which provides a means of assessing therapists’ mode use and clients’ preferences for and experience with certain communication modes based on the IRM (Taylor, 2008).The four versions of the CAM provide a reliable and valid assessment of client–therapist communication for use in real-time clinical practice and educational contexts.
Conclusion
This study presented evidence to support the psychometric properties of four versions of the CAM, using a heterogeneous clinical sample with a variety of rehabilitation issues. The results suggest that items on each mode subscale formed distinct unidimensional constructs reflecting unique approaches to therapeutic communication. These approaches, labeled therapeutic communication modes, are equally valued as viable approaches in therapy and conform to Taylor’s (2008) IRM. In addition, the CAM had satisfactory internal consistency, and it can be used by clients, therapists, and trained observer-raters to assess different perspectives of the therapeutic relationship. Undergoing rehabilitation for a neurological or orthopedic event involves a range of challenges for clients, many of which involve stressful interactions with therapists. The CAM has promise as a clinical assessment tool and also as a means of training and reflection in clinical fieldwork and other educational settings. Future study is recommended to reword a small number of items.
Footnotes
Acknowledgments
The authors thank Elena Espiritu, Jenica Lee, Kay McGee, and Supriya Sen, who helped screen patients and invited other therapists to participate in this study. We also extend our gratitude to all the clinical therapists and clients who were involved in the study.