Abstract
In response to the expanding needs of health services and limited budgets of health care systems, growing efforts have been made to optimize patients’ functional independence in acute care settings (Boninger et al., 2012; Gage et al., 2012; Kane et al., 1996; Kramer et al., 1997). In acute care settings, a high proportion of patients, particularly those with neurological disorders, have limitations in mobility, self-care, and cognition (Jette et al., 2014; Müller et al., 2011). Assessing limitations in these functional areas has become crucial work for occupational therapy practitioners throughout their care for these patients.
Previous studies have found that effective functional assessments enable rehabilitation practitioners to identify patients’ needs, determine care plans, establish rehabilitation goals, and even predict health outcomes (Johnston & Granger, 1994; Kramer, 1997). However, most of the existing functional assessments used in acute care settings have been criticized for being time consuming and designed only for specific patient populations or settings (Jette et al., 2009). There is a lack of standardized functional measures that are comprehensive, easy to administer, and adequate for use in tracking functional changes across individual patients with different functional levels and across diverse care settings (Andresen et al., 2000; Dobrzykowski, 1999; Kane et al., 2000).
In Taiwan, the most widely adopted functional measure in rehabilitation settings is the Barthel Index (BI; Mahoney & Barthel, 1965), and it has been widely used with patients with neurological disorders (Hsueh et al., 2001). The BI was originally designed to measure the functional independence of people in inpatient rehabilitation settings (Fricke & Unsworth, 2010), and it was determined to have sound psychometric properties (Chen et al., 2016; Park, 2018). Despite its wide use, the BI has been criticized for demonstrating floor and ceiling effects, lacking measurement precision, and not covering all relevant activity domains (Cohen & Marino, 2000; de Morton et al., 2008). In addition, the BI has been found to be less effective in people with higher functional levels and cannot be continually used as patients transition from an acute phase to a chronic phase (Katz, 2003).
A standardized functional assessment tool, the Activity Measure for Post-Acute Care (AM–PAC) “6-Clicks,” was developed to assess activity functions of patients in acute care rehabilitation settings (Haley, Coster, Andres, Ludlow, et al., 2004). The 6-Clicks are three inpatient short forms created from the AM–PAC item pool that measure three functional domains: Basic Mobility, Daily Activity, and Applied Cognition. This assessment tool has demonstrated advantages over other existing tools used in acute care settings in that it is easy and quick to administer and was developed using Item Response Theory, which enables scores derived from the 6-Clicks to be linked to scores derived from other AM–PAC short forms as a patient transitions across settings (Haley, Coster, Andres, Kosinski, & Ni, 2004).
Given the advantages of the 6-Clicks, interest in using this instrument has increased in practice worldwide, and it has become necessary to translate the instrument into multiple languages. Recently, the AM–PAC Outpatient Short Forms were translated into Mandarin (Chang et al., 2018). However, the 6-Clicks instrument had never been translated and validated in Chinese-speaking populations. Without such evidence, the instrument cannot be used in acute care settings in Taiwan, and cross-country comparisons are not possible.
The purpose of the current study was to develop a Mandarin version (the official Chinese dialect used by Taiwanese) of the 6-Clicks and examine its psychometric properties (including floor and ceiling effects, internal consistency, interrater reliability, test–retest reliability, and convergent validity) among patients with neurological disorders in acute care rehabilitation settings in Taiwan. We hypothesized that the Mandarin version of the AM–PAC 6-Clicks would have good reliability and sufficient validity.
Method
Participants
In total, 231 neurological patients participated in this study. Demographic characteristics of participants are shown in Table 1.
Participant Demographics (N = 231)
Note. MoCA scores are not adjusted for education level. Means of BI and MoCA scores are consistent with previous studies of people with acute stroke, Alzheimer’s disease, and mild cognitive impairment in Taiwan (Hsueh et al., 2001; Tsai et al., 2012). Therapists’ demographics were not included because they were not considered study participants and did not need to provide consent. AM–PAC = Activity Measure for Post-Acute Care; BI = Barthel Index; M = mean; MoCA = Montreal Cognitive Assessment; SD = standard deviation.
Instrument
The 6-Clicks is a validated measure designed to be used with patients in acute care settings regardless of the diagnosis (Haley, Coster, Andres, Ludlow, et al., 2004). It includes three subscales: Basic Mobility, Daily Activity, and Applied Cognition. Each subscale consists of six items that measure a patient’s capacity for functional activities, which occupational and physical therapists commonly assess in acute care settings (Jette et al., 2014). Each item is scored by an occupational therapist or physical therapist, through either direct observation of a patient performing an activity or clinical judgment about that patient’s probable capabilities, on a scale ranging from 1 (unable to do) to 4 (no difficulty at all) on the basis of the amount of difficulty the patient has in completing a task. The total score for each subscale ranges from 6 to 24 and can be transformed into standardized t scores. A lower score equates to a lower level of function. The Basic Mobility and Daily Activity subscales have demonstrated strong validity and interrater reliability in patients in acute care hospitals in the United States (Jette et al., 2014, 2015).
Adaptation
The dual-panel translation approach was adopted to translate the 6-Clicks. This approach was proven to be an effective and efficient method that showed advantages over the forward–backward translation method by generating not only linguistic equivalency but also conceptually and culturally adapted translations (McKenna et al., 2013; Swaine-Verdier et al., 2004). The translation process we used involved the following stages:
Stage 1—Bilingual panel translation: A bilingual panel consisting of six people who were fluent in both American English and Mandarin Chinese and who had resided in the United States for more than 5 yr worked as a team to produce translations through group discussions.
Stage 2—Lay panel adaptation: A lay panel, consisting of 10 Taiwanese with diverse demographic backgrounds (50% male; mean age = 51.5 yr, standard deviation = 19.4; 70% high school graduate or higher educational achievement) and who were neither patients nor linguistic professionals, reviewed and revised the translation produced by the bilingual panel to ensure the wording was adequate and comprehensible.
Stage 3—Cognitive debriefing interviews: A well-trained interviewer conducted cognitive interviews with 10 occupational and physical therapists from rehabilitation departments of three teaching hospitals by observing them completing the translated AM–PAC and asking them to describe how they had arrived at answers when they evaluated patients of the same demographic group for which the AM–PAC was developed, what challenges they perceived, and whether they found the items and response scales to be comprehensible and relevant.
The research team then reviewed the results derived from the cognitive interviews to determine whether they needed to make changes in the translation. Before making any changes, the research team confirmed with the AM–PAC developers that the changes would not affect the meaning of the items and instructions. The final version of the translated AM–PAC Inpatient Short Form was used for psychometric validation.
Validation
Participants and Procedures
This study was conducted between December 2015 and May 2016, and two participant groups were involved. Patient participants were recruited from rehabilitation departments of three teaching hospitals of a university in Taiwan with the following inclusion criteria: age ≥20 yr, medically stable, been diagnosed with a neurological disorder, and just been admitted to inpatient rehabilitation care and beginning to (or about to) receive occupational therapy or both occupational therapy and physical therapy. Patients were excluded if they were unconscious or confused or had unstable vital signs.
On the basis of the criteria, a convenience sample of 231 adults was recruited, and they consented to participate in the study. The research staff introduced the occupational therapists of these participants to the 6-Clicks, providing full instructions and training. The therapists were then asked to complete all three subscales of the instrument during their initial patient visit as routine documentation within the first week of their patients’ hospital admission.
Among the recruited participants, a convenience subgroup of 148 participants was invited for the test–retest reliability study. The occupational therapists (N = 18) of these participants were asked to rerate their patients’ activity limitations using the Mandarin version of the 6-Clicks within 5–7 days after the first assessment. For the interrater reliability study, a subgroup of 139 participants was selected to receive both occupational and physical therapy. In addition to the ratings by occupational therapists, these participants’ physical therapists (N = 22), who were also fully trained in using the instrument and were unaware of the occupational therapists’ ratings, were asked to rate their patients’ activity limitations during their first patient visits.
Written informed consent was obtained from all participants. The study protocols were reviewed and approved by the institutional review board of the university and all the recruitment sites.
Other Measures
Other data besides information from the Mandarin version of the 6-Clicks were collected as each participant was referred to rehabilitation. Sociodemographic characteristics, such as sex and age, and clinical data, such as diagnosis and date of onset of illness, were obtained from medical charts and face-to-face interviews by two well-trained research assistants. In addition, two legacy measures were used to examine the convergent validity of the AM–PAC Inpatient Short Form: the Mandarin version of the BI and the Mandarin version of the Montreal Cognitive Assessment (MoCA; Nasreddine et al., 2005). Both of these assessments were administered by the occupational therapists as they administered the AM–PAC to all participants admitted to the hospital.
The BI, which evaluates 10 basic activity of daily living items, including feeding, transferring, grooming, toileting, bathing, ambulating, stair climbing, dressing, bowel control, and bladder control, was adopted for comparison with the AM–PAC Basic Mobility and Daily Activity subscales. The total possible BI score ranges from 0 to 100. The Mandarin version of the BI showed adequate internal consistency and interrater reliability as well as strong correlation with FIM reliability and validity (Hsueh et al., 2001, 2002).
The Mandarin version of the MoCA, a 30-point clinician-administered brief cognitive assessment, was adopted for comparison with the AM–PAC Applied Cognition subscale. The MoCA assesses performance in eight cognitive domains (i.e., Short-Term Memory, Visuospatial Abilities, Executive Functioning, Attention, Concentration, Working Memory, Language, and Orientation to Time and Place). It has demonstrated high sensitivity and specificity to detect mild cognitive impairment or early-stage dementia (Hoops et al., 2009).
Statistical Analysis
Reliability
The internal consistency of the Mandarin version of the 6-Clicks was determined by Cronbach’s α•. The intraclass correlation coefficient (ICC) was used to examine test–retest reliability at the subscale level. A two-way analysis of variance was used to compute the variance needed to calculate the ICC (Shrout & Fleiss, 1979). Weighted • κ coefficients were used to assess the item-level reliability.
The subscale-level interrater reliability was also analyzed using ICC statistics and was used to examine the degree of agreement between a pair of occupational and physical therapists when they rated the same patient. The fixed effect of the ICC model was used to compute the ICC value for interrater reliability (Shrout & Fleiss, 1979). Weighted κ• coefficients were used to assess the item-level reliability.
Validity
To estimate the convergent validity of the Mandarin version of the 6-Clicks, we examined the strength of correlations between its Basic Mobility and Daily Activity subscale scores with the BI score and between its Applied Cognition score and the MoCA score, using Spearman’s correlation coefficient. To detect a correlation coefficient of •≥.30, a sample of 92 subjects was required to attain 90% power (Portney & Watkins, 2008).
Results
Distribution of 6-Clicks Scores
Among the 231 participants, 220 had complete data on the Mandarin version of the 6-Clicks; 11 participants had missing data because they did not schedule a first visit with their occupational therapist within the first week of their admission to the hospital. For the remaining 220 participants, mean t scores for the Basic Mobility, Daily Activity, and Applied Cognition subscales of the Mandarin version of the 6-Clicks were 32.37, 30.96, and 28.53, respectively (Table 2). Figure 1 shows the distribution of total scores on each subscale. Among these participants, 54 (24.54%), 62 (28.18%), and 53 (24.09%) obtained the minimum score of 6, and 15 (6.82%), 14 (6.36%), and 25 (11.36%) obtained the maximum score of 24 on the Basic Mobility, Daily Activity, and Applied Cognition subscales, respectively.
Test–Retest and Interrater Reliability
Note. AM–PAC = Activity Measure for Post-Acute Care; CI = confidence interval; ICC = intraclass correlation coefficient.
Distribution of scores in each domain of the Activity Measure for Post-Acute Care “6-Clicks.”
Reliability
For internal consistency, Cronbach’s α• coefficient of >.70 is considered adequate, and >.90 is considered excellent (Aaronson et al., 2002). Our results indicated that internal consistency was excellent for all three subscales of the Mandarin version of the 6-Clicks: Basic Mobility (α = .97), Daily Activity (α• = .97), and Applied Cognition (α• = .98).
The test–retest and interrater reliability results are shown in Table 2. ICC and weighted κ• values of >.8 indicate excellent reliability, and values in the range of .6–.8 represent adequate reliability (Bushnell et al., 2001). Our findings indicated that the test–retest reliability was excellent for all subscales (ICCs = .97–.98). The item-level reliability was also excellent (κ• coefficients = .82–.98). The interrater reliability was excellent for all subscales (ICCs = .94–.96). The item-level reliability was adequate to excellent (κ• coefficients = .73–.94).
Validity
A correlation coefficient of >.60 indicates a strong correlation; .30–.60, a moderate correlation; and <.30, a poor correlation (Andresen, 2000). Our results demonstrated that correlations of both Basic Mobility and Daily Activity with the BI were strong (r = .73 and .72, respectively, p < .001). The correlation between Applied Cognition and the MoCA was also strong (r = .82, p < .001).
Discussion
This is the first study to translate the 6-Clicks into Mandarin and examine its psychometric properties among patients with neurological disorders in acute care rehabilitation settings. The dual-panel translation approach, as described in earlier studies (Chang et al., 2018, 2019), was demonstrated to be an effective and efficient translation approach, which led to this conceptually and linguistically sound translation of the 6-Clicks Inpatient Short Form. Findings of this study provide psychometric evidence for use of this instrument by both clinicians and researchers to assess activity limitations among patients with neurological disorders in rehabilitation settings.
Comparing the psychometric properties of the original English-language 6-Clicks (Jette et al., 2014) with those of the Mandarin version showed that this version has higher floor effects (i.e., 2.7% for Basic Mobility and 2.3% for the Daily Activity forms) but lower ceiling effects (i.e., 15.5% for Basic Mobility and 10.1% for the Daily Activity forms) than the original. These inconsistencies might have been the result of differences between patient characteristics in the two samples.
In the U.S. study (Jette et al., 2014), only 12.2% of patients had neurological disorders, whereas in our study, all patients were diagnosed with neurological disorders, such as stroke, brain injury, or spinal cord injury. Patients with these neurological conditions are likely to have more severe dysfunctions than patients with orthopedic or other conditions when they are admitted to acute settings. Therefore, the initial scores on the 6-Clicks of patients with neurological disorders are likely to be at the lower end and may increase after rehabilitation.
The reliability estimates identified in the current study are comparable with reliability estimates found in previous studies. The internal consistencies of the three subscales of our translated 6-Clicks (.97–.98) were slightly higher than the internal consistencies of the original (.91–.96; Jette et al., 2014). The test–retest reliabilities of our version (.97–.98) were similar to those for the Mandarin version of the AM–PAC Outpatient Short Form (ICC = .98; Chang et al., 2018) and higher than those for the Brazilian Portuguese version of the AM–PAC (.88–.95; Souza et al., 2018). The interrater reliabilities measured by weighted κ• values of item agreement in our study were higher than those for the original (Jette et al., 2014) and similar to those of the Dutch version of the 6-Clicks Basic Mobility subscale (Geelen et al., 2018), indicating that the Mandarin version is a reliable measure that can be recommended for use in acute rehabilitation settings.
For validity, the Mandarin version of the Basic Mobility and Daily Activity subscales showed high correlations with the BI (.71–.72), similar to results of a previous study that compared the Mandarin version of the Outpatient Short Form with the BI (.68–.70; Chang et al., 2018). Our study results were also similar to those of a study that compared the 6-Clicks with the FIM (Jette et al., 2014), in which the correlation between the AM–PAC Basic Mobility and the FIM Motor scores was .69 and that between the AM–PAC Daily Activity and the FIM activities of daily living scores was .65. These findings are consistent with our expectations because both the BI and FIM are designed to measure mobility and daily activities of patients in acute rehabilitation as does the 6-Clicks. However, the BI and FIM do not include lower level activities (e.g., rolling in bed, sitting up from bed), whereas the 6-Clicks does. Therefore, the 6-Clicks is correlated with but not completely identical to the other two measures.
The Mandarin version of the 6-Clicks Applied Cognition subscale showed a high correlation with the Mandarin version of the MoCA, higher than the correlation between the Applied Cognition subscale of the Mandarin version of the AM–PAC Outpatient Short Form and the Mini-Mental State Examination (MMSE; Chang et al., 2018). Although both the MoCA and the MMSE are designed to assess cognitive impairment, the MoCA covers a broader range of cognitive functions, including executive functions, higher level language abilities, and complex visuospatial processing, which could be closer to the content assessed by the AM–PAC Applied Cognition subscale (Nasreddine et al., 2005).
In Taiwan, an urgent need exists for practitioners to have a universal functional assessment tool to monitor patients’ functional status and progression across the continuity of rehabilitation services from hospitals to the community. The AM–PAC is a conceptually sound measurement tool that can be used across rehabilitation care settings. The AM–PAC Outpatient Short Form has been translated into Mandarin and introduced to rehabilitation practice in Taiwan. By linking this Outpatient Short Form with the Mandarin version of the 6-Clicks, patients’ functional abilities in different activity domains can be continually monitored as they transition across treatment settings, which will be critical data for service providers when planning treatment and resource allocation.
Limitations and Future Research
This study had several limitations. First, a convenience sample was used in this study, which may have limited the generalizability of the findings to patients in other settings or communities. A second limitation is that our study sample involved only patients with neurological disorders, and the majority of them had a diagnosis of stroke, which in fact reflects the actual distribution of patient groups in acute rehabilitation settings in Taiwan. We also did not include patients who were unconscious or confused, because they were unable to consent to participate in the study.
Future studies may need to replicate this study with a larger and more diverse sample of patients in acute rehabilitation with different conditions. Last, although this study provides preliminary support for the use of the Mandarin version of the 6-Clicks in acute care rehabilitation settings, we did not examine the responsiveness, standard error of measurement, minimal detectable change, or minimal important difference of this version. Further examination of the aforementioned properties is needed to fully demonstrate the psychometric properties and utility of the Mandarin version of the 6-Clicks.
Implications for Occupational Therapy Practice
The Mandarin version of the 6-Clicks demonstrated strong internal consistency, test–retest reliability, interrater reliability, and convergent validity in a Chinese-speaking population.
Findings of this study confirm the appropriateness of the use of the Mandarin version of the 6-Clicks with patients in acute rehabilitation, making it a valuable addition to validated measures available in Taiwan for use by occupational therapists.
Conclusion
The findings of this study provide evidence for the reliability and validity of the three subscales of the Mandarin version of the 6-Clicks. The instrument demonstrated its potential capability to accurately and reliably assess functional abilities among patients with neurological disorders in acute rehabilitation settings in Taiwan. Further work needs to be conducted to provide more evidence of the validity and responsiveness of the instrument.
Footnotes
Acknowledgments
We especially acknowledge Alan Jette, of the MGH Institute of Health Professions, Boston, for generous feedback throughout this project. We thank members of our research team for their contributions: Yi-Cian Lai and Yi-An Lien (Taipei Medical University). We thank the collaborating hospitals: Shuang Ho Hospital, Wan Fang Hospital, and Taipei Medical University Hospital. This study was funded by the Ministry of Health and Welfare, Taiwan (Grants MOHW104-TDU-B-212-113001 and MOHW105-TDU-B-212-133018), and in part by the Ministry of Science and Technology, Taiwan (Grants MOST105-2628-B-038-003-MY3 and MOST108-2314-B-038-030-MY3), Ministry of Education, Taiwan (Grant DP2-108-21121-01-A-01-01), and the National Health Research Institutes, Taiwan (Grant NHRI-EX108-10819PC).