Clinical Utility of the Combined Use of the Canadian Occupational Performance Measure and Goal Attainment Scaling

Abstract

OBJECTIVE. We investigated the clinical utility of combined use of Goal Attainment Scaling (GAS) and the Canadian Occupational Performance Measure (COPM) to plan goals and measure progress in a community rehabilitation setting.

METHOD. Fourteen participants with traumatic brain injury completed an outpatient, goal-directed 12-wk occupational therapy program; 53 goals were generated. Performance and satisfaction self-ratings and GAS ratings were collected before and after intervention. Self-awareness, motivation to change, and perceived client-centeredness measures were taken before intervention.

RESULTS. Sensitivity to change was demonstrated by significant improvements after intervention for total performance self-ratings on the COPM and GAS T scores.

CONCLUSION. Combined use of these tools, although time consuming, resulted in goals that were perceived almost unanimously as client centered, despite most participants' having moderate or severe impairment in self-awareness. The process also enabled subjective and objective demonstration of goal achievement, thereby supporting the clinical utility and treatment validity of the combined use of these tools.

Keywords

brain injuries goals occupational therapy outcome assessment (health care)patient-centered care

The use of goals to facilitate a client-centered approach is essential to rehabilitation (Barnes & Ward, 2000). Concerns over goal planning include difficulty objectively measuring goal performance (Kuipers, Foster, Carlson, & Moy, 2003) and the perception that goal planning is time consuming (Playford et al., 2000) and increases workload (MacLeod & Macleod, 1996). Individual differences between the goals and expected achievement levels of clients also pose methodological challenges to ensure validity when goal-based outcomes are used to evaluate interventions (Ottenbacher & Cusick, 1993). In the rehabilitation of people after traumatic brain injury (TBI), additional challenges to realistic goal planning include cognitive impairment, reduced motivation, and diminished self-awareness (Fischer, Gauggel, & Trexler, 2004; Kuipers, Carlson, Bailey, & Sharma, 2004). We evaluated the combined use of two established goal-based assessments in a community-based occupational therapy program with clients with TBI.

Two widely recognized, validated goal-based assessments incorporate features of goal-based planning with objective measurement: the Canadian Occupational Performance Measure (COPM; Law et al., 1990) and Goal Attainment Scaling (GAS; Kiresuk & Sherman, 1968). The COPM is an individualized, client-centered, client-rated assessment tool based on the Canadian Occupational Performance Model (Law et al., 1990). It is designed for use with people of all ages with various disabilities and backgrounds; its use with people with TBI has been documented (Jenkinson, Ownsworth, & Shum, 2007). The validity and reliability of the COPM have been widely established (Carswell et al., 2004).

GAS offers a framework for documenting individualized goals in a quantifiable manner and was originally developed for use in the mental health field (Kiresuk & Sherman, 1968). It has been used to measure outcomes of brain injury rehabilitation programs (Joyce, Rockwood, & Mate-Kole, 1994), including home-based brain injury rehabilitation settings (Lannin, 2003). GAS has reported high interrater reliability and good concurrent validity with other outcome measures (Malec, 1999). Goal attainment scaling involves calculation of a standard T score, which enables comparison of performance between groups and individuals, provided the predictions made at initial evaluation and assessments made at follow-up evaluations are reliable (Ottenbacher & Cusick, 1990).

The combined use of the COPM and GAS has the potential to overcome some of the challenges to the use of goal planning and measurement in TBI rehabilitation. Specifically, whereas the COPM is designed to facilitate client-centered goal setting and evaluate goal achievement in accordance with clients’ perception of their performance and satisfaction, GAS enables objective measurement of goal achievement. The use of both tools in planning goals and measuring progress may be more time efficient than using a long battery of multiple-outcome measures that lack the capacity to measure progress toward specific goals (Chiu & Tickle-Degnen, 2002). In particular, both instruments may efficiently capture progress at the activity and participation performance levels, which are characteristic of the types of goals set in community programs (Kuipers et al., 2003).

Goal-based measures, such as the COPM and GAS, may also offer greater ecological validity than neuropsychological tests in which the relationship between test scores and real-life functioning is unclear (Shordone & Long, 1996). Tools such as the COPM and GAS enable measurement of performance of real-life, meaningful, individualized goals that may not be achievable using standardized tests (Wilson, 2003). The COPM and GAS have both been shown to be more responsive to change than several standardized outcome measures (Jenkinson et al., 2007; Rockwood et al., 2003; Rockwood, Joyce, & Stolee, 1997). Although sensitivity to change is one aspect of clinical utility, we aimed to investigate the measures’ clinical utility more generally.

Clinical utility can be defined as “the value of a measure, relative to other measures, for a particular purpose” (Haynes & O’Brien, 2000, p. 314). It consists of several elements, including sensitivity to change, cost-effectiveness, user friendliness, and treatment validity. Treatment validity is specifically concerned with the degree to which an instrument contributes to the conceptualization of the client’s problem, treatment planning or effectiveness, and the measurement of the treatment process or outcome (Haynes & O’Brien, 2000). We used the COPM and GAS in combination to enable identification of client-centered goals and to monitor goal performance from the perspective of the person with TBI and the treating occupational therapist. Although aspects of the clinical utility of both measures have been previously reported (Trombly, Radomski, Trexel, & Burnet-Smith, 2002), the clinical utility of their combined use has not been investigated.

Previous investigations into the clinical utility of the COPM have found it to be sensitive to change in outpatient brain injury rehabilitation (Trombly et al., 2002); inpatient neurological rehabilitation (Bodiam, 1999); and community-based, brain injury–specific group rehabilitation programs (Jenkinson et al., 2007). Its use is associated with a sense of goal ownership and satisfaction with progress by clients and their significant others (Doig, Fleming, Cornwell, & Kuipers, 2009). In other populations, client-centered goal setting using the COPM has been shown to increase participation in the goal-formulation process and perceived ability to manage personal and domestic activities of daily living (ADLs) after rehabilitation (Wressle, Eeg-Olofsson, Marcusson, & Henriksson, 2002).

People with reduced self-awareness and cognitive impairment, however, tend to make inflated self-ratings of their performance using the COPM (Ownsworth, McFarland, & Young, 2002), and questions remain about correlations among COPM self-ratings, awareness, depression, and cognitive function (Jenkinson et al., 2007). The current study provided an opportunity to explore the COPM’s sensitivity and treatment validity by investigating perceived client-centeredness of goals and subsequent goal achievement for people with TBI with different levels of self-awareness in a community-based occupational therapy program.

GAS has been found to be sensitive to change in general rehabilitation (Rockwood et al., 2003) and community-based neurological rehabilitation (Lannin, 2003). Despite the complexity of the GAS procedure, which requires adequate time and skills to break goals down into appropriately sized steps to accurately reflect progress (Eames, Ward, & Siddons, 1999), the time commitment involved has not been widely researched.

Combined use of the COPM and GAS has been shown to result in clinically significant improvements in self-identified goals (Trombly et al., 2002). More research is needed to support the clinical utility of this approach to goal setting and evaluation in TBI. Unique to this study is the investigation into clinical utility and treatment validity of the combined use of the COPM and GAS in a community-based, one-to-one occupational therapy program for people with TBI.

Aims

The first aim was to investigate the clinical utility of the combined use of the COPM and GAS by (1) describing the effectiveness of GAS at operationalizing the COPM goal areas, (2) determining their relative sensitivity in measuring change, (3) determining agreement between participants and their significant others on perceived change in COPM performance as well as correspondence with the measured change in GAS scores, and (4) examining the time required to use the COPM and GAS.

The second aim was to investigate the treatment validity of the combined use of both measures by (1) determining the perceived client-centeredness of goals and (2) describing differences in level of self-awareness, motivation to change, and perceived client-centeredness of goals between those who achieved clinically significant change on GAS and the COPM and those who did not.

Method

Research Design

This clinical utility study involved a pretest–posttest design with within-groups comparisons and descriptive analysis (Stein & Cutler, 2000).

Participant Selection

Participants were 14 people with TBI living in the community who had been recently discharged from inpatient brain injury rehabilitation in Brisbane, Queensland, Australia, and their 14 significant others. Inclusion and exclusion criteria are outlined elsewhere (Doig et al., 2009). All participants were receiving outpatient, clinic-based rehabilitation at a large metropolitan hospital. In addition to their regular outpatient rehabilitation, participants received as part of their involvement in this research project a 12-wk program of goal-directed occupational therapy.

Instruments

The COPM is a semistructured interview tool designed to identify problems with occupational performance across self-care, productivity, and leisure (Law et al., 1990). The client identifies difficulties with specific tasks in these three areas and rates the importance of each identified task on a scale ranging from 1 (not important at all) to 10 (extremely important). Occupational performance and satisfaction with performance on each goal are also rated by the client on a scale ranging from 1 to 10. Change over time in performance and satisfaction with performance on each goal are deemed to be clinically significant when a pre- to postintervention change of ≥2 points occurs (Law et al., 1994). Total COPM performance and satisfaction scores can be yielded by dividing the sum of performance or satisfaction scores by the number of goals. In the current study, significant others also rated the client’s occupational performance on the COPM using the 1-to-10 scale.

GAS is not norm referenced; rather, it is an individualized measure designed to track within-subject longitudinal change (Ottenbacher & Cusick, 1993). It uses an ordinal measurement scale that requires the user to break down goal performance on a 5-point scale ranging from –2 to 2. Table 1 shows the goal attainment levels and example goals for 2 participants. A T score can be calculated that represents a person’s performance on his or her GAS goals. The GAS T score has a mean (M) of 50 and a standard deviation (SD) of 10 and can be calculated using the following formula:

where W_i is the assigned goal weight and X_i is the outcome for each behavior (i.e., value from –2 to 2; Ottenbacher & Cusick, 1990). The r value represents the estimated intercorrelation for the outcome scores that can be safely assumed to be .30 (Kiresuk & Sherman, 1968) and is a constant in the formula. T scores of 50 indicate expected levels of performance, and T scores above and below 50 indicate above- and below-expected levels of performance.

The Client-Centeredness of Goal Setting (C–COGS) Questionnaire assesses self-perception of importance and meaningfulness of goals on a scale ranging from 1 (not important) to 5 (totally important) and level of agreement with the statement “the goals are what I want to work on” on a 1-to-5 scale. The C–COGS questionnaire was also used to assess how meaningful the significant others felt the goals were to the participants as well as how strongly they felt the participants desired to work on the chosen goals.

Table 1.

Two Example Goals Scaled Using Goal Attainment Scaling and Corresponding Performance Levels

Performance Levels	Goal: Legible Handwriting	Goal: Taking a Taxi Cab Independently
Much more than expected (2)	Fills out standard form with 100% legibility	Takes cab to unfamiliar destination independently
Somewhat more than expected (1)	Fills out standard form with 75%–99% legibility	Takes cab to familiar destination independently
Expected level of performance (0)	Fills out standard form with 50%–74% legibility	Takes cab to familiar destination (1%–33% assistance)
Somewhat less than expected (–1)	Fills out standard form with 25%–49% legibility	Takes cab to familiar destination (34%–66% assistance)
Much less than expected (–2)	Fills out standard form with <25% legibility	Takes cab to familiar destination (>66% assistance)

The Self Awareness of Deficits Interview (SADI) is a structured interview designed to assess intellectual awareness of physical, cognitive, behavioral, and interpersonal difficulties (Fleming, Strong, & Ashton, 1996). The three SADI indexes include self-awareness of deficits, self-awareness of the functional implications of deficits, and ability to set realistic goals. Each is scored on a scale of 0 to 3, giving a total score of 9. Lower scores indicate higher levels of self-awareness. Total SADI scores were used to categorize participants into groups with no or mild disorder (score = 0–3), moderate disorder (score = 4–6), and severe disorder (score = 7–9) to assist with descriptive analysis. The SADI Relative’s Checklist was used to collect collateral data on the client’s performance to assist with assigning SADI scores.

The Change Awareness Questionnaire (CAQ; Lam, McMahon, Priddy, & Gehred-Schultz, 1988) is a 24-item scale adapted for use with people with TBI and is based on Prochaska and Di Clemente’s (1982) Stages-of-Change Model. The CAQ identifies three stages of change: (1) precontemplation (low self-awareness and low motivation to change), (2) contemplation (considering change), and (3) action (the person is actively changing). The CAQ score reflects a person’s motivation to change. Each item is rated on a 5-point scale ranging from strong disagreement (1) to strong agreement (5). Scores range from 8 to 40 for each subscale. By comparing subscale scores, a person’s stage of motivation to change can be determined.

Procedures

Ethical clearance was obtained from the relevant ethics committees before the study began. Before participation in goal setting or commencement of therapy, the SADI and CAQ were administered. The COPM was then used to identify specific occupational performance goals. Guidelines by Ottenbacher and Cusick (1990) were used to develop goal attainment scales for each goal area according to the following seven steps:

Identification of objectives and problem areas. The COPM (Law et al., 1990) was administered by a trained user within the week after discharge from inpatient rehabilitation. Interviews involved the participants and their significant others. The therapist prompted participants using task analysis (Pedretti & Wade, 1996) and motivational interviewing techniques (Miranti & Heinemann, 2004) to break broader, long-term goals into specific problem areas to be targeted in a 12-wk program.

Identification of specific goal behaviors for each problem area was achieved by assessment and discussion. For example, for a handwriting problem, the therapist would clarify whether the goal was to improve legibility or writing speed after observing writing.

Determination of methods for goal measurement. Procedures for measuring performance on each goal were documented using objective and measurable terms to ensure a consistent approach. For example, the goal of taking a taxi, outlined in Table 1, included a written list of 11 possible steps, and the percentage of assistance required for taking a taxi was calculated by dividing the number of steps requiring assistance by the total number of steps.

Selection of expected level of performance was determined by observational assessment and input from the participants, their significant others, and the hospital occupational therapist.

Identification of the levels of performance. Current level of function (set at either –1 or –2 depending on the level of function) for each goal was determined by observational assessment and in consultation with the participant’s hospital occupational therapist. Most and least favorable levels were scaled around the expected level of performance.

Review of goals was completed to ensure equal gaps between levels, to avoid instances of performance being rated at more than one level, and to avoid performance falling between levels.

Weighting each goal to enable calculation of GAS T score. Participant’s COPM importance ratings were used to rank goals in order of priority. When a participant identified four goals, the most important goal was weighted 4, and the least important goal weighted 1. When goals were equally important, all were given the same weight.

After completion of goal planning, the C–COGS was administered to determine the importance and perceived client-centeredness of the goals. Pre- and postintervention GAS ratings were assigned by a researcher not involved in delivering the therapy and were measured through observational assessment. When observation was not possible, reports of performance by the participant, significant other, and treating occupational therapist were gathered, and a rating was decided on by consensus. Pre- and postintervention COPM participant ratings of performance and satisfaction as well as significant other COPM ratings of performance on each goal were collected.

The time taken to administer the COPM, scale goals (including liaison to obtain information), document goals using the GAS, and perform goal assessment using the COPM and GAS was recorded in field notes by Emmah Doig throughout the assessments.

Data Analysis

Total COPM scores and GAS T scores were calculated, and sensitivity to change was examined by comparing preintervention and postintervention scores using paired t tests. Before analysis, the normality of the distributions was examined by calculating kurtosis and skewness and by visually examining quantile–quantile plots and histograms. Data for COPM self-rated satisfaction postintervention were negatively skewed because of one outlier; removal of this outlying data resulted in a near-normal distribution. Therefore, paired t tests on this variable were calculated with this participant’s data excluded. Data for all other variables approximated a normal distribution.

COPM and GAS preintervention and postintervention raw scores were examined, and the percentage achieving clinical significance was calculated. Change scores for self-rated COPM performance, significant other COPM performance, and GAS scores were tabulated, and percentage of agreement on the direction of change (improved, no change, or decline) between the measures was calculated. Average times to administer the COPM and develop the GASs were summarized. Scores for the C–COGS scale were reported. The types of goals generated using the COPM and the effectiveness of GAS in operationalizing the goal areas were examined by tabulating the COPM problem areas and corresponding GAS and reviewing research field notes, which summarized goal planning and review sessions.

An independent research assistant reviewed the field notes, summarizing common themes about goals. Participants achieving and not achieving clinically significant improvement after intervention on the COPM and GAS were divided into two groups, and the total SADI scores, CAQ Contemplation and Action subscale scores, and C–COGS scores for each group were compared using descriptive statistics. Participants were divided according to severity of impairment of self-awareness, and differences in goal attainment were descriptively analyzed.

Results

Participants were 12 men and 2 women ages 18–57 (M = 27.4, SD = 10.7). The mean initial Glasgow Coma Scale (Teasdale & Jennett, 1974) score was 6.2 (SD = 3.9), and the average length of posttraumatic amnesia was 85.5 days (SD = 51.1, missing = 2), indicating a very severe level of injury. Mean length of acute hospital stay was 64.8 days (SD = 37.4), and inpatient rehabilitation was 134.2 days (SD = 109.4). On discharge from inpatient rehabilitation, the average Disability Rating Scale (Rappaport, Hall, Hopkins, Belleza, & Cope, 1982) score was 5.5 (SD = 1.6), and the average FIM^® (Hamilton, Granger, Sherwin, Zielezny, & Tashman, 1987) score was 110.0 (SD = 9.2). Participants’ significant others were a parent (n = 11) or spouse–partner (n = 3). All participants resided with their significant others after discharge from inpatient rehabilitation.

Client-Generated Goals and Corresponding GAS

Table 2 gives some examples of priority occupational performance areas identified and the corresponding behaviors used to scale GAS to objectively measure progress.

Fifty-three goal attainment scales were developed for the 14 participants. The goals were categorized as leisure (7), self-care (6), domestic ADLs (8), community ADLs (2), study-related goals (8), functional use of the upper limb (6), cognitive function (6), work-related skill building (1), driving-related skill building (2), and speed or legibility of handwriting (7). Using the International Classification of Functioning, Disability and Health (World Health Organization [WHO], 2001), only 1 of the GAS goals could be classified under the impairment level; 46 were activity-level goals, and 6 were participation-level goals.

All participants generated their own goals during administration of the COPM with the exception of Participant 12 and Participant 13, who were unable to identify problem areas during administration of the COPM. In both cases, a therapist-directed approach (Leach, Cornwell, Fleming, & Haines, 2010) was used in which participants agreed that the therapist could facilitate goal planning by consulting with their significant other and inpatient occupational therapist to assist with identifying problem areas. The problem areas were used to identify potential goals, which were discussed with the participants, agreed to be worked on in the program, and subsequently rated using the COPM.

Table 2.

Examples of Priority 1 Occupational Performance Areas (COPM) and Corresponding Behaviors Used to Measure GAS Goal Performance

Participant No.	COPM Occupational Performance Area	GAS Goal
1	Being able to play pool	1. Performance of pool—percentage of accuracy hitting ball 2. Participation—number times per week plays pool
3	Doing more hobbies and leisure activities	Number of times per week initiates participation in a new or old hobby for own leisure
4	Concentrating better in a lecture situation	Amount of time able to pay attention to an audiovisual presentation (i.e., documentary-style TV program) and percentage of questions able to be answered correctly about the content
5	Making coffee (previous occupation, barista)	Independence, including use of adaptive equipment or compensatory techniques (i.e., use of nonaffected hand where he normally would not); number of steps requiring assistance
8	Handwriting faster and more legibly	1. Percentage of legibility of letters 2. Speed of handwriting (timed)
9	Putting on trousers independently	Percentage of steps able to do self (used FIM to break down steps of lower-body dressing)
12	Remembering to shower	Number of prompts daily (consistency of performance over a week)
15	Using right hand more normally to eat and drink	Use of adaptive equipment or techniques, number of compensatory movements and smoothness of movement to 1. Cut food using a knife and fork and 2. Drink from a cup using right hand

Note. COPM = Canadian Occupational Performance Measure; GAS = Goal Attainment Scaling.

Sensitivity of GAS and the COPM

The COPM and GAS data were analyzed for the 14 participants who had completed their interventions, with the exception of self-rated satisfaction data for 1 participant, which was excluded from the analysis to achieve a more normal distribution. One participant (Participant 2) decided soon after completion of goal planning to cease all therapies and therefore dropped out of the study before beginning the intervention. Group means and SDs for pre- and postintervention COPM scores and GAS T scores are presented in Table 3.

Table 3.

Group Means for Preintervention and Postintervention GAS T Scores and COPM Standard Scores (Using Paired t Tests)

Measure	Preintervention T Score M (SD)	Postintervention T Score M (SD)	t
GAS	36.9 (6.3)	52.8 (6.2)	–9.65**
COPM
Performance (client)	5.0 (1.9)	8.0 (1.5)	–5.07**
Satisfaction (client)	4.6 (1.8)	8.5 (1.1)	–6.40**
Performance (significant other)	4.3 (1.5)	7.6 (1.2)	–7.92**

Note. GAS = Goal Attainment Scaling; COPM = Canadian Occupational Performance Measure; M = mean; SD = standard deviation. Clinically significant change defined as ≥2-point change on COPM and achievement of expected level of performance ≥50 on GAS.

p < .01 (two-tailed).

A review of each participants’ pre- and postintervention COPM scores in each of the 51 occupational performance problem areas indicated clinically significant improvements (defined as an increase of ≥2 points) in COPM self-rated performance for 34 goals and COPM self-rated satisfaction for 38 goals. For most of the remaining goals, performance was rated on the COPM as improved but was not clinically significant (n = 6) or was rated as equivalent to or less than preintervention but not significantly less (n = 6). A review of pre- and postintervention GAS scores (n = 53) indicated clinically significant improvement (defined as achievement of ≥0 during the intervention period) on 44 goals, improvement from –2 to –1 on 6 goals, no change in performance level on 2 goals, and decline in performance level for 1 goal. Table 3 also shows statistically significant improvements in GAS T scores, COPM performance and satisfaction self-ratings, and significant other ratings of performance between the pre- and postintervention assessments using paired t tests (p < .01).

Change Agreement for GAS and Participant and Significant Other COPM Performance Ratings

Goal performance on COPM and GAS ratings showed general agreement between the measures (i.e., most showed improvement in both GAS and at least one COPM rating). Exceptions were Participant 1 (improvement in GAS but not COPM performance), Participants 6 and 10 (improvements on COPM but not in GAS performance), and Participant 13 (improvement on significant-other COPM performance only).

For 70% of the goals, there was agreement between participants and their significant others on the direction of change in COPM performance, which was always in the direction of improvement and corresponded with improvements in the objective GAS rating. On review of the occasions on which COPM self-rating and significant other rating did not agree in terms of direction of change, most occasions were accounted for by 2 participants. For Participants 1 and 13, self-rated performance did not correspond with significant other COPM performance ratings or GAS ratings for all of their goals. These participants perceived that they had not changed or had worsened after intervention. In both cases, all significant other ratings and most GAS ratings showed an improvement in performance, although Participant 13 did not achieve the GAS expected level of performance. Both participants fell into the severe category of impairment of self-awareness at admission to the program, according to the SADI. Participant 13 also demonstrated symptoms of depression. The SADI was repeated with Participant 13 with his permission, and his self-awareness had improved after intervention (from total SADI = 7 to 3).

Time Required to Use the COPM and GAS

It took, on average, 4.5 hr per participant to plan and document goals in preparation for the rehabilitation program. This preparation involved establishing goal areas using the COPM (1 hr), liaison with necessary others (30 min), observational assessment to establish goal scales (1 hr 45 min), and writing goal attainment scales (1 hr 25 min). It took approximately 1 hr per participant to reassess performance using the COPM and GAS, including preparation of materials for reassessments. Liaison with others was greatest for Participants 12 and 13; throughout the program, writing of GASs took less time with practice, facilitated by a growing bank of varied GAS goals.

Degree of Client-Centeredness

The C–COGS questionnaire was completed with 11 of the 14 participants (not completed with Participants 10 and 11 because of time constraints and not completed with Participant 12 because of severe memory deficit). Twelve significant others completed the C–COGS questionnaire, because in two cases, the significant other was not involved in the goal-planning process. Goals were either totally or mostly important to participants (M = 4.4, SD = 0.50) and participants desired to work on them (M = 4.1, SD = 1.31). All significant others agreed that the goals were what they wanted their relative to work on (M = 4.5, SD = 1.17). Most significant others indicated that the goals were what their relative wanted to address (M = 4.2, SD = 1.36), with the exception of the significant others of Participants 12 and 13, who indicated the goals were neither important to their relatives nor goals their relatives wished to attain.

Self-Awareness, Change Motivation, and Perceived Client-Centeredness on Goal Attainment

All participants were either in the contemplation or action stage of change. Mean scores for CAQ subscales were 18.6 (SD = 5.6) for precontemplation, 30.8 (SD = 2.8) for contemplation, and 31.3 (SD = 3.5) for action. According to the SADI, 4 participants demonstrated severe, 5 demonstrated moderate, and 5 showed no or mild impairment of self-awareness (overall M = 4.6, SD = 2.8).

Figure 1 indicates that the amount of improvement in performance after intervention, according to the change in GAS T scores, was similar, on average, when participants were grouped according to severity of self-awareness of deficits. Participants who achieved expected levels of performance on GAS (n = 11) and clinically significant change on COPM self-rated performance (n = 10), satisfaction (n = 11), and COPM significant other–rated performance (n = 10) were characterized by various levels of impairment to self-awareness and motivation to change. Participants who did not achieve expected levels of performance on GAS T scores (n = 3) still made clinically significant improvements after the intervention, and 2 of those participants also demonstrated clinically significant improvements on COPM self-ratings and significant other COPM ratings. Only 1 participant (Participant 13) did not achieve clinically significant change according to COPM self-ratings and achieved below-expected levels of GAS performance after intervention. This participant demonstrated poor levels of client-centeredness on his goals as well as depression and severe impairment of self-awareness.

Figure 1.

Average change in Goal Attainment Scaling (GAS) T scores before and after intervention, by severity of deficit in self-awareness (according to Self Awareness of Deficits Interview).

Discussion

In relation to clinical utility, the findings indicate that GAS was an effective tool for operationalizing the range of client-centered goals facilitated by administration of the COPM. The GAS was useful in translating the broad goal areas generated by the COPM into specific behavioral actions. This finding lends weight to the treatment validity of combined use of the GAS and COPM in directing and structuring the content of therapy and elucidating client aspirations in behavioral terms (Doig et al., 2009). The nature of the goals varied, but most were at the level of activity and participation, which is consistent with previous studies investigating the nature and content of goals for people with brain injury, including TBI, undergoing rehabilitation in the community (Kuipers et al., 2003; Phipps & Richardson, 2007). The GAS T score and COPM total scores and raw scores were found to be sensitive measures reflecting clinically significant change after intervention on individual goals. This result supports previous findings that the COPM (Bodiam, 1999; Jenkinson et al., 2007, Trombly et al., 2002) and GAS (Lannin, 2003; Rockwood et al., 1997) are sensitive measures of change after rehabilitation for people with TBI.

In relation to agreement between significant other and self-ratings, there was general agreement in terms of the direction of improvement after intervention when comparing COPM performance scores. This finding is notable given that most participants presented with moderate or severe levels of impairment of self-awareness. Because it has previously been suggested that people with impaired self-awareness may be more likely to underreport impairments (Sherer et al., 1998), one might have expected there to be less agreement about change (i.e., most participants perceiving no change or decline because of inflated self-ratings of performance at preintervention assessment). Although findings indicate consistency between self-ratings and significant other perception of change direction on COPM performance, case-by-case analysis indicated that 2 participants (1 and 13) self-rated their performance on the COPM as having declined between preintervention and postintervention assessments, whereas their relatives rated their performance as improved and objective GAS assessment indicated improvements. In those cases, the COPM did not show the clinically significant improvement, which was evident on observational assessment, because these participants tended to rate their performance higher on admission than did their significant others.

Given that the participants presented with severe impairment of self-awareness and the intervention specifically involved developing self-awareness through provision of feedback about performance of the goal-related activities, it is likely that self-awareness may have improved over time and that postintervention COPM ratings became more realistic. This finding highlights that in some cases, self-rating tools such as the COPM may not reflect positive change; however, objective assessment and collateral reports do indicate change. This finding was not consistent, however, because all other participants demonstrating moderate and severe deficits in self-awareness on preintervention assessment presented with congruency in change direction between their COPM self-rating and significant other self-ratings as well as GAS change scores.

It is clear that multiple factors are relevant to people with TBI and could potentially affect the reliability of self-ratings of performance, including cognitive impairment, poor self-awareness, and emotional distress (Ownsworth et al., 2002). This finding highlights the value of a comprehensive assessment process incorporating observational assessment in conjunction with self-rating and collateral ratings of performance to reliably determine change. This approach is also a necessity when using GAS, because not all goals can be assessed by means of observation because of practical and time constraints. Moreover, for participation-level goals in which performance is “involvement in a life situation” (WHO, 2001, p. 10), an observational assessment at a single point in time will not capture the frequency of participation and will not measure consistency of performance over time. Such instances require self-report and collateral reports from family or others.

The COPM and GAS in combination are clinically useful for several reasons. Goal attainment scaling addresses the issue of subjectivity inherent in relying solely on self-ratings of performance, because in most cases, it enables an objective functional assessment to be made in client-generated goal areas. When used in combination with GAS, the COPM captures those aspects of performance that are not quantifiable and may only be known by the participant and significant other (i.e., quality of performance and satisfaction with performance) given prior knowledge of the performance standards, environment, and situation unique to each individual. Moreover, the use of the client’s goal importance rating at assessment points formalizes the process of allocating a weighting to the goals in calculating the GAS T score, should the client’s goal importance ratings change over time.

Although the time requirement for the COPM and GAS varied across participants, it was mostly invested in the goal-planning process. We found that a therapist-directed approach required more time than a client-centered approach because of increased liaison time to gather collateral information for those participants who could not report problem areas as a basis for goal planning. Preparation for assessment of goal performance also involved a significant amount of time, primarily because individualized, contextually relevant goals by nature often require individualized materials to assess goal performance. The time involved in using the COPM and GAS to plan and document goals and to assess goal performance needs to be weighed against the benefit and treatment validity of having individually tailored goals to direct rehabilitation.

In relation to treatment validity, the findings indicate that participants made clinically significant progress toward goals on one or both of the COPM or GAS and almost unanimously perceived their goals as client centered. This finding is noteworthy because most participants demonstrated moderate or severe impairment of self-awareness, and only 58% were in the contemplation stage of change. Interestingly, several participants were in the contemplation stage of change even though they were drawn from a group of clients who had already received considerable amounts of rehabilitation. Moreover, those who were unable to identify problems with occupational performance and generate their own goals did not perceive their goals as client centered. Those who generated their own goals were more likely to want to work on the goals and felt the goals were important to them.

Client-centeredness and participation in goal planning has previously been reported to result in better outcomes (Wressle et al., 2002). The two participants whose significant others indicated that they had little desire to work on their goals were also the only participants who were unable to generate their own program goals. One, Participant 12, presented with moderate impairment of self-awareness and severe memory impairment; the other, Participant 13, presented with severe impairment of self-awareness and depression. On completion of the program, Participant 12 perceived less overall improvement than did his significant other but made clinically significant improvements on GAS. Participant 13 perceived a decline in performance despite the relative’s report of clinically significant improvements and despite objective improvement of GAS scores. These findings highlight that low levels of perceived client-centeredness of goals, impaired self-awareness, and low motivation are not necessarily barriers to the achievement of outcomes provided the person engages in the therapy program. Involvement may be contingent on family and therapists using a therapist-led approach with increased investment of time to collaborate and negotiate to set goals, have clients sign up to work on goals, and encourage participation in rehabilitation.

The main limitation of the study is the small sample size limiting statistical power; however, the analysis was complemented by descriptive statistics and in-depth case-specific information aiding clinical interpretation of results. Qualitative findings reported in a companion article—which explored the participants, their significant others, and their occupational therapists’ experiences and opinions of the goal-planning process and goal-directed therapy—provide further insight into the treatment validity of the COPM and GAS (Doig et al., 2009).

Conclusion

The COPM and GAS, used in combination, were sensitive to change and useful in planning, documenting, and measuring progress on client-centered goals from participants’ and therapists’ perspectives in the context of a short-term community-based occupational therapy program. The COPM and GAS enabled formulation of client-centered goals resulting in high levels of perceived client-centeredness. The benefits of using the COPM to develop and monitor goals that are meaningful to the client, together with the GAS to operationalize goals and objectively measure change, substantially outweigh the time and other costs.

Footnotes

Acknowledgments

We thank the participants and their significant others for their participation. We acknowledge the hard work of the occupational therapists delivering the therapy programs. We thank the Princess Alexandra Hospital Foundation and the Queensland Health Community Rehabilitation Research Scheme for providing funding to undertake this study.

References

Barnes

M. D.

, & Ward

A. B.

(2000). Textbook of rehabilitation medicine. Oxford, England: Oxford University Press.

Bodiam

(1999). The use of the COPM for the assessment of outcome on a neurorehabilitation unit. British Journal of Occupational Therapy, 62, 123–126.

Carswell

McColl

M. A.

Baptiste

Law

Polatajko

, & Pollock

(2004). The Canadian Occupational Performance Measure: A research and clinical literature review. Canadian Journal of Occupational Therapy, 71, 210–222.

Chiu

, & Tickle-Degnen

(2002). Learning from evidence: Service outcomes and client satisfaction with occupational therapy home-based services. American Journal of Occupational Therapy, 56, 217–220.

Doig

E. J.

Fleming

Cornwell

, & Kuipers

(2009). Qualitative exploration of a client-centered, goal-directed approach to community-based occupational therapy for adults with acquired brain injury. American Journal of Occupational Therapy, 63, 559–568.

Eames

Ward

, & Siddons

(1999). Clinical audit of the outcome of individualised occupational therapy goals. British Journal of Occupational Therapy, 62, 257–260.

Fischer

Gauggel

, & Trexler

L. E.

(2004). Awareness of activity limitations, goal setting, and rehabilitation outcome in patients with brain injuries. Brain Injury, 18, 547–562. doi: 10.1080/02699050310001645793

Fleming

J. M.

Strong

, & Ashton

(1996). Self-awareness of deficits in adults with traumatic brain injury: How best to measure. Brain Injury, 10, 1–15. doi: 10.1080/026990596124674

Hamilton

B. B.

Granger

C. V.

Sherwin

F. S.

Zielezny

, & Tashman

J. S.

(1987). A uniform national data system for medical rehabilitation. In Fuhrer

(Ed.), Rehabilitation outcomes: Analysis and measurement (pp. 137–147). Baltimore, MD: W. H. Brookes.

10.

Haynes

S. N.

, & O’Brien

W. H.

(2000). Principles and practice of behavioural assessment. New York: Kluwer Academic.

11.

Jenkinson

Ownsworth

, & Shum

(2007). Utility of the Canadian Occupational Performance Measure in community-based brain injury rehabilitation. Brain Injury, 21, 1283–1294. doi: 10.1080/02699050701739531

12.

Joyce

B. M.

Rockwood

K. J.

, & Mate-Kole

C. C.

(1994). Use of Goal Attainment Scaling in brain injury in a rehabilitation hospital. American Journal of Physical Medicine and Rehabilitation, 73, 10–14.

13.

Kiresuk

T. J.

, & Sherman

R. E.

(1968). Goal Attainment Scaling: A general method for evaluating comprehensive community mental health programs. Community Mental Health Journal, 4, 443–453. doi: 10.1007/BF01530764

14.

Kuipers

Carlson

Bailey

, & Sharma

(2004). A preliminary exploration of goal-setting in community-based rehabilitation for people with brain impairment. Brain Impairment, 5, 30–41. doi: 10.1375/brim.5.1.30.35404

15.

Kuipers

Foster

Carlson

, & Moy

(2003). Classifying client goals in community-based ABI rehabilitation: A taxonomy for profiling service delivery and conceptualizing outcomes. Disability and Rehabilitation, 25, 154–162. doi: 10.1080/0963828021000024898

16.

Lam

C. S.

McMahon

B. T.

Priddy

D. A.

, & Gehred-Schultz

(1988). Deficit awareness and treatment performance among traumatic head injury adults. Brain Injury, 2, 235–242. doi: 10.3109/02699058809150947

17.

Lannin

(2003). Goal Attainment Scaling allows program evaluation of a home-based occupational therapy program. Occupational Therapy in Health Care, 17, 43–55. doi: 10.1300/J003v17n01_04

18.

Law

Baptiste

Carswell

McColl

M. A.

Polatajko

, & Pollock

(1990). Canadian Occupational Performance Measure. Ottawa, ON: CAOT Publications.

19.

Law

Baptiste

Carswell

McColl

M. A.

Polatajko

, & Pollock

(1994). Canadian Occupational Performance Measure manual (2nd ed.). Toronto, ON: CAOT Publications.

20.

Leach

Cornwell

Fleming

, & Haines

(2010). Patient centered goal setting in a subacute rehabilitation setting. Disability and Rehabilitation, 32, 159–172.

21.

MacLeod

G. M.

, & Macleod

(1996). Evaluation of client and staff satisfaction with a goal planning project implemented with people with spinal cord injuries. Spinal Cord, 34, 525–530.

22.

Malec

J. F.

(1999). Goal Attainment Scaling in rehabilitation. Neuropsychological Rehabilitation, 9, 253–275. doi: 10.1080/096020199389365

23.

Miranti

V. S.

, & Heinemann

A. W.

(2004). Systematic motivational counseling in rehabilitation settings. In Miles Cox

M. W.

& Klinger

(Eds.), Handbook of motivational counseling New York: Wiley.

24.

Ottenbacher

K. J.

, & Cusick

(1990). Goal attainment scaling as a method of clinical service evaluation. American Journal of Occupational Therapy, 44, 519–525.

25.

Ottenbacher

K. J.

, & Cusick

(1993). Discriminative versus evaluative assessment: Some observations on goal attainment scaling. American Journal of Occupational Therapy, 47, 349–354.

26.

Ownsworth

T. L.

McFarland

, & Young

R. M.

(2002). The investigation of factors underlying deficits in self-awareness and self-regulation. Brain Injury, 16, 291–309. doi: 10.1080/02699050110103986

27.

Pedretti

L. W.

, & Wade

I. E.

(1996). Therapeutic modalities. In Pedretti

L. W.

(Ed.), Occupational therapy: Practice skills for physical dysfunction (pp. 293–319). St. Louis, MO: Mosby/Year Book.

28.

Phipps

, & Richardson

(2007). Occupational therapy outcomes for clients with traumatic brain injury and stroke using the Canadian Occupational Performance Measure. American Journal of Occupational Therapy, 61, 328–334.

29.

Playford

E. D.

Dawson

Limbert

Smith

Ward

C. D.

, & Wells

(2000). Goal-setting in rehabilitation: Report of a workshop to explore professionals’ perceptions of goal-setting. Clinical Rehabilitation, 14, 491–496. doi: 10.1191/0269215500cr343oa

30.

Prochaska

J. O.

, & Di Clemente

C. C.

(1982). Transtheoretical therapy: Toward a more integrative model of change. Psychotherapy: Therapy, Research, and Practice, 19, 276–288.

31.

Rappaport

Hall

K. M.

Hopkins

Belleza

, & Cope

D. N.

(1982). Disability rating scale for severe head trauma: Coma to community. Archives of Physical and Medical Rehabilitation, 63(3), 118–123.

32.

Rockwood

Howlett

Stadnyk

Carver

Powell

, & Stolee

(2003). Responsiveness of goal attainment scaling in a randomized controlled trial of comprehensive geriatric assessment. Journal of Clinical Epidemiology, 56, 736–743. doi: 10.1016/S0895-4356(03)00132-X

33.

Rockwood

Joyce

, & Stolee

(1997). Use of Goal Attainment Scaling in measuring clinically important change in cognitive rehabilitation patients. Journal of Clinical Epidemiology, 50, 581–588. doi: 10.1016/S0895-4356(97)00014-0

34.

Sherer

Boake

Levin

Silver

B. V.

Ringholz

, & High

W. M.

Jr. (1998). Characteristics of impaired awareness after traumatic brain injury. Journal of the International Neuropsychological Society, 4, 380–387.

35.

Shordone

R. J.

, & Long

C. J.

(1996). The ecological validity of neuropsychological testing. Delray Beach, FL: GR Press/St. Lucie Press.

36.

Stein

, & Cutler

S. K.

(2000). Clinical research in occupational therapy (4th ed.). San Diego, CA: Singular.

37.

Teasdale

, & Jennett

(1974). Assessment of coma and impaired consciousness: A practical scale. Lancet, 2(7872), 81–84.

38.

Trombly

C. A.

Radomski

M. V.

Trexel

, & Burnet-Smith

S. E.

(2002). Occupational therapy and achievement of self-identified goals by adults with acquired brain injury: Phase II. American Journal of Occupational Therapy, 56, 489–498.

39.

Wilson

B. A.

(2003). Goal planning rather than neuropsychological tests should be used to structure and evaluate cognitive rehabilitation. Brain Impairment, 4(1), 25–30. doi: 10.1375/brim.4.1.25.27030

40.

World Health Organization. (2001). International classification of functioning, disability and health. Geneva: Author.

41.

Wressle

Eeg-Olofsson

A. M.

Marcusson

, & Henriksson

(2002). Improved client participation in the rehabilitation process using a client-centred goal formulation structure. Journal of Rehabilitation Medicine, 34, 5–11. doi: 10.1080/165019702317242640