Abstract
Keywords
Hematological cancers, such as multiple myeloma, lymphoma, and leukemia, affect many Americans. In 2002 through 2006, age-adjusted incidence for myeloma was 5.6 per 100,000 population per year, lymphoma was 22.3 per 100,000, and leukemia was 12.2 per 100,000 (Horner et al., 2009). Stem cell transplantation (SCT) is now standard therapy for many of these hematological cancers, and >17,000 SCTs are performed each year in the United States (Goldman & Ausiello, 2008). SCT involves eradicating the abnormal bone marrow and blood cells through chemotherapy, then replacing them with healthy blood stem cells that settle in the bone marrow, essentially creating a new immune system (Stewart, 2006). The new cells can come from the patient (an autologous transplant) or a related or unrelated donor (an allogeneic transplant).
The transplant process is debilitating, and patients experience side effects such as nausea, vomiting, oral mucositis, skin erythema, and fatigue (Appelbaum, 2003). After stem cell reinfusion, patients need to avoid crowds and sources of bacteria to prevent life-threatening infection at a time when the immune system is not fully functional. Research has shown that physical recovery from SCT can take up to 1 yr, and emotional and vocational recovery is often more prolonged (Baker, Zabora, Polland, & Wingard, 1999; Syrjala et al., 2004; Wingard, 1994). Researchers are interested in describing outcomes of SCT in terms of return to normal activity routine but have reported a lack of instruments to measure recovery in this fashion (Lee et al., 2001). The Activity Card Sort (ACS; Baum & Edwards, 2001) could fill this void and provide a gauge of the patient’s progress in resuming occupations after this aggressive anticancer therapy.
The ACS was developed to measure levels of activity engagement in older adults. The tool uses a Q sort method (Cordingley, Webb, & Hiller, 1997) in which respondents sort 80 cards depicting people engaged in various activities into mutually exclusive categories, such as never done and do now. The ACS yields a total score and four domain scores. The domains include Instrumental Activities (e.g., driving, paying bills, child care), Low-Physical-Demand Leisure (e.g., puzzles, quilting, photography), High-Physical-Demand Leisure (e.g., bicycling, woodworking, hiking), and Social Activities (e.g., volunteer work, visiting with friends, traveling).
One advantage of the ACS is that it is a client-centered measure of participation. The response categories ask a person to indicate whether an activity is part of his or her routine. If the activity is not one that the respondent wants or needs to do, then that activity does not get factored into the final score. This process is in contrast to other measures of function, such as the Medical Outcomes Study SF–36 (Ware et al., 2007), in which respondents are scored in terms of their activity limitations (e.g., being unable to ascend a flight of stairs) regardless of whether each activity is a necessity for them in their home or community.
The original ACS (which is similar to the recently released second edition; Baum & Edwards, 2008) has three versions. The institutional version can be used with adults in hospitals or nursing facilities. The community version can be used with adults living in their homes. The recovering version measures changes in activity engagement after experiencing an illness or injury. The recovering version scoring algorithm yields an intuitively meaningful number that reflects the percentage of activities that have been retained during recovery. The manual suggests that the recovering version of the ACS can be used longitudinally to track a patient’s progress in returning to a previous level of function after a health event (Baum & Edwards, 2001), such as SCT.
Although the ACS manual indicates that the tool can be used longitudinally to describe recovery, a literature search of four databases yielded no studies that have used the tool in this way (CINAHL, Medline, Web of Science, and PsycINFO). The recovering version asks respondents to compare their current participation with their participation before an event. Essentially, it taps into a sense of whether an activity is a part of a patient’s “normal” routine. It is an interesting empirical question to consider the stability of that recollection over a period of time. Adding further complexity, stem cell reinfusion is a distinct, salient event, yet the period preceding reinfusion may be a slow process of functional decline caused by the disease or treatment before SCT. This process is in contrast to a hip fracture or cerebral vascular accident in which activity engagement might be dramatically and vividly different when comparing before and after event activities.
Because the tool gives a client-centered and detailed account of activity resumption, we used a modification of the ACS (ACSm) in a longitudinal pilot study describing recovery after SCT. The ACSm was included with measures of quality of life, performance status, fatigue, and cognitive status. These data offer a way to explore some of the psychometric properties of the ACSm when used longitudinally. Accordingly, this secondary analysis answers three questions: (1) How consistent (i.e., stable and reliable) are participants’ designations of their “previously done” activities over an 18-mo period? (2) To what degree is the ACS internally consistent? (3) Do ACSm scores correlate with other established measures of performance status and quality of life? The first question addresses the reliability of one aspect of the tool when used longitudinally and the second and third questions address the internal consistency and the construct validity of the tool for this population.
Method
Participants
Participants were recruited from the bone marrow transplantation program at a National Cancer Institute–designated comprehensive cancer center. The study was approved by the Committee for the Protection of Human Subjects at Dartmouth College. Patients were eligible for the study if they were > age 18, scheduled to undergo an SCT, and able to provide informed consent. Thirty-six participants enrolled in the study and completed at least one modified ACS.
Measures
Activity Card Sort (Modified).
Researchers have explored the psychometric properties of the ACS when used with college students, adults, and older adults without illness and with people with multiple sclerosis, cerebral vascular accident, and Alzheimer’s disease (Baum & Edwards, 2001; Everard, Lach, Fisher, & Baum, 2000; Katz, Karpin, Lak, Furman, & Hartman-Maeir, 2003; Sachs & Josman, 2003). The authors reported a 1-wk test–retest reliability coefficient of .90 for the recovery score (Baum & Edwards, 2001), and other researchers reported similar results for 14- and 30-day test–retest reliability (Carpenter et al., 2007; Chan, Chung, & Packer, 2006; Everard et al., 2000; Katz et al., 2003). Internal consistency of the four subscales ranged from α = .61 to .83 (Katz et al., 2003). Construct validity has been supported in that the tool is able to distinguish between healthy and ill populations (Katz et al., 2003), and Baum (1995) found that retained activities are related to decreased caregiver burden and increased independence in self-care activities for people with Alzheimer’s disease. A score of 100 indicates that the person has resumed his or her previous level of activity engagement, and a score of 0 indicates that a person is no longer doing any of his or her previous activities, nor has he or she added new activities to the current routine.
Although the ACS was designed as a manual card sort, Everard and colleagues (2000) used a paper-and-pencil version of the tool in a mail survey study. They compared the checklist with the community version card sort in a pilot sample of 20 community-dwelling older adults. They found acceptable concurrent validity between the versions (r > .72 for all subscales) and high test–retest reliability for the checklist version at an interval of 74 days (r > .83 for all subscales).
This study used a modification of the recovering version of the ACS in a self-administered checklist format. The original response choices were as follows: not done prior to illness; continued to do during illness; given up due to illness; beginning to do again; or new activity since illness. Initially, the word illness was replaced with transplant. After the first 6 respondents had completed their second ACSm, discussions with these respondents indicated that further wording changes were needed. First, regarding the given up category, respondents said sometimes they had not completely stopped doing an activity but were doing it far less than usual or that they had not been doing the activity because of something other than the transplant (e.g., they do not garden in the winter). Second, when used longitudinally, the beginning to do again response became problematic. For example, someone may have suspended an activity for the first month after transplant and begun doing the activity again at 1 mo. If asked to do the ACSm at 3 mo when she or he was back to doing the activity as she or he used to, there might not be a logical response to choose. That is to say, the person had not “continued to do the activity since transplant” yet was not currently “beginning to do it again.” After consultation with the author (Baum, personal communication, October 18, 2005), the following categories were used to address the aforementioned problems: never done prior to transplant; do now (as often as before transplant); do less or differently than before transplant; not done since transplant; or new activity since transplant.
Scoring followed the original algorithm of the ACS. The number of activities “done previously” is a count of all activities that the person endorsed with any of the do now.., do less.., or not done since…categories. To calculate current activities, activities endorsed by the do now.., do less.., and new activity…categories were awarded 1, 0.5, and 1 points per activity, respectively, then summed. To calculate the percentage of activities retained, the sum of the current activities is divided by the number of activities done previously.
Karnofsky Performance Scale.
The Karnofsky Performance Scale (KPS; Karnofsky, Burchenal, & MacLeod, 1949) is a 10-point, clinician-rated measure of how much a disease or treatment is affecting a patient’s ability to move and complete activities of daily living. Higher scores indicate higher performance ability. The KPS demonstrates adequate interrater reliability at r = .89 (Schag, Heinrich, & Ganz, 1984). Construct validity is supported by associations with other measures of daily function (Buccheri, Ferrigno, & Tamburini, 1996; Schag et al., 1984) and the ability of the scale to accurately predict survival time (Buccheri et al., 1996). In this study, KPS scores were documented as part of the standard procedure in the oncology service by the treating clinician and were obtained by medical record review. As a gross measure of performance status, the KPS scores should be positively correlated with the ACSm scores.
Functional Assessment of Cancer Therapy–General and Cognitive Scales.
The Functional Assessment of Cancer Therapy–General (FACT–G) is a 27-item measure of self-reported, health-related quality of life (Cella et al., 1993). The FACT–G provides an overall score in addition to scores of four subscales: Physical Well-Being (seven items), Social/Family Well-Being (seven items), Emotional Well-Being (six items), and Functional Well-Being (seven items). Higher scores indicate better quality of life. The FACT–G has a 1-wk test–retest reliability of r = .92. Internal consistency reliability of the subscales ranges from α = .69 (Social/Family Well-Being) to α = .82 (Physical Well-Being) with an overall internal consistency of α = .89 for the entire 27 items. Authors reported that construct validity of the FACT–G is supported by relationships with mood, performance status, and another quality-of-life measure. Moreover, the FACT–G was able to distinguish between groups known to differ on stage of disease, performance status, and need for emotional support. In the current analysis, the FACT–G was expected to be moderately correlated with the ACS, given that the ACS has been found to be related to quality of life in other populations (Chan et al., 2006; Doney & Packer, 2008; Hartman-Maeir, Soroker, Ring, Avni, & Katz, 2007). The Functional Well-Being subscale, in particular, is a measure of a patient’s satisfaction with ability to work, manage the home, and enjoy life and should be positively correlated with the ACS.
In addition, the FACT–G can be augmented by disease- or condition-specific subscales. One such subscale is the Cognitive Function subscale (FACT–Cog; Jacobs, Jacobsen, Booth-Jones, Wagner, & Anasetti, 2007). The 33-item FACT–Cog itself has four subscales: Perceived Cognitive Impairment, Impact of Perceived Cognitive Impairment on Quality of Life, Comments From Others, and Perceived Cognitive Abilities. These subscales are not combined but instead are scored and interpreted separately. The FACT–Cog was included in the study to track the subtle cognitive changes often reported by people who have had chemotherapy (Ahles et al., 2002; Saykin, Ahles, & McDonald, 2003). Because these changes are not typically as profound as those seen in dementia, the changes were not expected to dramatically influence a patient’s activity engagement (although they may decrease a patient’s enjoyment of or proficiency in certain activities). As such, the Perceived Cognitive Impairment subscale was used as a way to explore the divergent validity of the ACSm. ACSm and FACT–Cog scores were not expected to be highly correlated with each other.
Procedure
Participants were invited to enroll in the study when they were scheduled to undergo SCT. All the measures were self-administered surveys delivered by mail. Confidentiality was protected by assigning identification numbers to each participant, and files were stored in locked cabinets and password-protected computer databases. Participants completed the FACT scales before their stem cell reinfusion and completed both the ACSm and the FACT–G and FACT–Cog scales in one survey packet at 1 mo, 3 mo, 6 mo, 9 mo, 12 mo, and 18 mo after stem cell reinfusion. Twenty-nine participants completed the first post-SCT survey at 30 days after stem cell reinfusion. Seven participants did not complete the first 30-day assessment (most often because of feeling unwell), but they did complete the next scheduled survey at 3 mo after stem cell reinfusion. These 36 participants are used to answer Research Question 3, regarding construct validity. Medical records were reviewed at each assessment point to collect the clinician-rated KPS score for that time period.
Twenty-nine participants completed the ACSm at least twice, allowing their ACSm scores to be analyzed for reliability (Research Questions 1 and 2). Of those 29 participants, 5 completed two surveys, 7 completed three surveys, 6 completed four surveys, 3 completed five surveys, and 8 completed all six surveys containing an ACSm. This analysis was conducted as an interim analysis during data collection to ascertain whether the ACSm would provide reliable and valid data for the pilot study.
Analysis
Reliability
The first research question pertains to the stability of participants’ categorization of an activity as “previously done” over an 18-mo period (i.e., an aspect of test–retest reliability). The first step in scoring the recovering version of the ACS is to categorize an activity as previously done or not done previously. Participants who endorsed do now (as often as before transplant), do less or differently than before transplant, or not done since transplant were coded as 1 for that activity item (i.e., the activity was done previously). Participants who endorsed never done prior to transplant or new activity since transplant were coded as 0 for that activity item (i.e., the activity was not previously done). This procedure yielded binary data that were collected at up to six time points. This designation of an activity as done previously (or not) should not change over the course of data collection because respondents are reflecting on and describing their participation in the fixed, unchanging past.
Next we coded each item for each participant as consistent in their “done previously” designation or inconsistent. Participants were consistent for an item if they gave the same response at each assessment point. For example, if a person indicated that the activity was “done previously” at each time point, then he or she was consistent. Likewise, if the person always reported that the activity was “not previously done,” then he or she was consistent. If, however, the person indicated that the activity was “done previously” on five of the six assessments but indicated that the activity was not previously done for one of the six assessments, then that participant was inconsistent for that item. We then calculated the proportion of items for which each participant was consistent. Each participant, therefore, had a proportion score, and descriptive statistics and 95% confidence intervals were generated for the sample. This process was first conducted looking at all 80 items, then replicated for each domain.
To explore internal consistency reliability (Question 2), we calculated Cronbach’s α for the total score and each domain at each time point. We expected that internal consistency would at least be greater than α = .60 for each domain, mirroring what has been previously reported in studies of the ACS (Baum & Edwards, 2008).
Validity
The third research question pertains to the validity of the ACSm scores. The data offered a way to assess the convergent and divergent validity of the ACSm. The former aspect was explored by looking at the correlation between the ACSm scores and the KPS score. Similarly, the Functional Well-Being domain of the FACT–G is a measure of self-rated ability to do one’s valued activities. Because both the KPS and the Functional Well-Being scales purport to measure aspects of a patient’s functional performance, moderate, positive correlations were expected (r between .50 and .75; Colton, 1974). In addition, functional performance as measured by the ACS has been found to be related to quality of life in other samples (Chan et al., 2006; Doney & Packer, 2008; Hartman-Maeir et al., 2007). Accordingly, we expected the ACSm scores to be positively and moderately associated with quality of life as measured by the FACT–G. The total ACSm scores for participants at their first completed assessment (n = 29 at 1 mo and n = 7 at 3 mo) were correlated with the KPS scores, the FACT–G, and the Functional Well-Being subscale of the FACT–G for that same assessment. To explore divergent validity, the ACSm scores were correlated with the FACT–Cog subscale. A negligible association was expected (r < .25; Colton, 1974). An α level of p < .05 was established a priori. Analyses were conducted with SAS 9.1 (SAS Institute, Inc., Cary, North Carolina).
Results
Demographics of the sample are presented in Table 1. The sample was White and non-Hispanic and evenly split between men and women; mean age was 54 (SD = 12). Most of the sample was college educated and married and had received an autologous SCT (i.e., the reinfused stem cells were taken from the patient’s own body before high-dose chemotherapy to eradicate the disease).
Participant Characteristics (N = 36)
Stem cells were collected from the person who has the disease.
Stem cells were donated by another person who did not have the disease.
Reliability
The first analysis considered the reliability of the previously done designations over the course of the longitudinal data collection. Table 2 contains summary statistics for the proportion of items for which participants consistently reported an activity as previously done or not done previously. On average, participants gave stable responses for 71.9% of the items. Proportions ranged from a low of reporting consistently on 42.5% items and a high of reporting consistently on 91.3% of items. The 95% confidence interval indicates that the average value for the population on the total score falls somewhere between 66.5% and 77.3%.
Summary Statistics for the Proportion of Items for Which Participants Were Reliable in Their Designation of an Activity as Previously Done Versus Not Done Previously on the Activity Card Sort (Modified; N = 29)
The highest stability of response was seen for the Instrumental Activities domain, with consistency on an average of 81.6% items. The Leisure domains demonstrated the lowest stability, with consistency on an average of 64.3% of items for high-demand leisure and 67.4% of items for low-demand leisure. The Social domain yielded an average of 76.1% of items for which participants were consistent and reliable in their reporting of an activity as done previously or not.
Internal consistency values are presented in Table 3. Internal consistency of the total score ranged from α = .86 at Time 4 (9 mo after SCT) to .96 at Time 2 (3 mo after SCT). Most values were greater than α = .70; however, each domain exhibited one value that fell below α = .70.
Internal Consistency for the Total Score and Each Domain of the Activity Card Sort (Modified) at Each Time Point
Validity
Pearson’s correlation coefficients between the ACSm and the other measures are presented in Table 4. As expected, activity retention as measured by the ACSm was correlated with performance status, quality of life, and functional well-being (r = .42, r = .51, and r = .59, respectively, all with p ≤ .011). There was no association between activity retention perceived cognitive impairment (r = .22, p = .19).
Construct Validity of Activity Card Sort (Modified; ACSm; N = 36)
Note. ACSm = Activity Card Sort (modified); FACT–Cog = Functional Assessment of Cancer Therapy–General and Cognitive Scales Cognitive Function subscale; FACT–G = Functional Assessment of Cancer Therapy–General and Cognitive Scales.
n = 35 because of missing data for 1 participant on one subscale.
Discussion
The purpose of this analysis was to confirm the construct validity and explore aspects of reliability for the ACSm when used longitudinally with survivors of SCT. To our knowledge, the consistency analysis we conducted has not been done with other populations. On one level, the tool asks respondents to classify whether or not an activity was part of their pretransplant repertoire. Although the scores on the ACSm are expected to change over time as a person recovers from SCT, this designation of an activity as done previously or not done previously should remain constant. In this sample, participants demonstrated sizable stability in their responses, with perfect consistency on an average of 72% of the items in the overall ACSm. Our approach to this analysis was conservative in that just one inconsistent response in the series of assessment points was enough to deem the response pattern “inconsistent.” Given that random error at any point in data collection could affect the reliability, it is encouraging that, on average, participants were responding consistently to almost three-quarters of the items on the tool. This finding suggests that memory and report of pretransplant activity repertoire is somewhat stable and that the tool can reliably be used to monitor activity resumption over time.
Participants demonstrated the highest reliability on the Instrumental domain and the lowest on the Leisure domains. One plausible reason could be that the Instrumental Activities domain generally reflects daily activities done often and on a regular basis. By contrast, the Leisure domains may contain activities that a person does somewhat infrequently. For example, a person could report not previously playing board games, yet if doing an assessment 1 wk after visiting with a nephew and playing a game of checkers, he or she might report playing board games as much as ever (maybe once or twice a year when the nephew visits). Both answers might feel accurate to the respondent on the basis of recent events. For activities that are more frequently done (presumably, dealing with laundry or dishes from the Instrumental domain or talking on the phone or going to church from the Social domain), there may be a clearer and firmer perception of whether the activity is typically part of one’s repertoire.
In terms of internal consistency, the values generally mirrored those reported when used with other populations. For example, Katz and colleagues (2003) reported αs of .66 and .61 for low-demand and high-demand leisure, respectively, and αs of .82 and .80 for instrumental and social activities, respectively. Sachs and Josman (2003) reported αs ranging from .70 to .90 for the domains in their factor analysis of the scores of an older adult population.
Regarding construct validity, the hypotheses were supported in that activity retention (ACSm) was related to performance status (KPS), quality of life (FACT–G), and specifically, the Functional Well-Being subscale of the quality of life instrument. The correlation between the ACSm and KPS might be expected to be higher because the KPS measures performance status. However, the KPS is a gross measure, with 10 possible responses, and there was not a great deal of variation in the scores (2 participants had a score of 70, 21 participants had a score of 80, and 13 participants had a score of 90). This finding needs to be acknowledged when interpreting the findings because the reduced variation could affect the size of the correlation. The results echo other studies that have shown that role retention is related to quality of life for survivors of SCT (Baker, Curbow, & Wingard, 1991) and that activity engagement as measured by the ACS is related to other measures of performance and quality of life (Chan et al., 2006; Doney & Packer, 2008).
It is also intuitively understandable that activity engagement demonstrated a weak and nonsignificant correlation with perceived cognitive impairment. This weak relationship suggests that participants in this sample with fewer perceived cognitive impairments had resumed more activities. Research has shown that some survivors of SCT may experience mild impairments in cognition (Beglinger et al., 2007), particularly in the areas of alertness, attention, memory (Poppelreuter, Weis, Mumm, Orth, & Bartsch, 2008), or reaction time (Schulz-Kindermann et al., 2007). These impairments may not prevent someone from resuming or continuing to do an activity but may affect the experience of the activity (e.g., satisfaction, enjoyment, length of time an activity takes, need for compensatory strategies). The ACSm measures activity retention or resumption, not the experiential variables, and that could explain the expected weak and nonsignificant correlation. However, the FACT–Cog results should be viewed as tentative, exploratory conclusions. We had decided to use the first assessment from each participant to maximize statistical power and allow inclusion of data from 36 participants. If we correlate only the FACT measures and the ACSm scores from the 30-day assessment (n = 29), then the correlation increases to r = .40 (p = .03).1 The validity analysis, therefore, suggests that the ACSm is demonstrating logical relationships with other measures, but further research should be undertaken to clarify what is seen in the data.
The following limitations of the study need to be taken into account when interpreting the results. Our modifications (i.e., the response choices and the choice to use a checklist as opposed to the manual card sort that includes pictures of the activities) and the small sample limit the generalizability of the findings. In addition, not every participant provided data at each assessment point. Most often, participants missed the earlier assessments because of feeling too unwell to complete the assessment, but others provided no reason. The results, therefore, cannot be seen as conclusive, yet they do provisionally support the reliability and validity of the ACSm when used longitudinally and encourage future use and study in this population.
Conclusion
The ACSm can be a useful client-centered clinical and research tool because it provides a measure of activity retention in addition to domains covering leisure, instrumental, and social activities. Although the reference point of the recovering version is a patient’s pre-illness event activity pattern, the ACS does not presume that return to the premorbid pattern is a goal for every person. The scoring allows for adoption of new activities to be weighted equally with premorbid activities and, therefore, provides an advantage over Likert scales that ask stem cell recipients to indicate how much their life has “returned to normal” and how much they are able to enjoy their “normal activities” (Lee et al., 2001). As a clinical tool, the ACS can be used to identify areas to target with rehabilitation efforts and to measure progress. As a research tool, the ACS can provide a measure of the pace and pattern of activity resumption after an illness event. This analysis provides provisional evidence that supports the reliability and validity of this modification of the ACS when used in a self-administered checklist format longitudinally to describe activity resumption after SCT. More research is warranted to determine whether the stability in the designation of an activity as “done previously” is similar to that seen in other populations, especially those with a more dramatic before and after illness event experience.
Footnotes
Acknowledgments
We appreciate the study collaborators: Lynn Root, Idalina Williams, Diane Stearns, Anna Schaal, and Elizabeth Kimtis. We also thank Colleen DeSchamp for her assistance with data management. This study was funded by a large project grant from the Hitchcock Foundation, Lebanon, NH.
The FACT–G and FACT–functional well-being correlations are essentially unchanged at r = .55, p = 0.0025, and r = .62, p = .0003, respectively, for the 30-day assessment.