A Pilot Study for Remote Evaluation of Upper Extremity Motor Function After Stroke: The Arm Capacity and Movement Test (ArmCAM)

Development of the ArmCAM

The Arm Capacity and Movement Test (ArmCAM) was designed to assess body functions and activities according to the International Classification of Functioning, Disability and Health (ICF). To develop the ArmCAM, two phases for assessment development were conducted (Bar-Shalita et al., 2009). In Phase 1, test construction and establishment of content validity for the domain of UE motor function were performed. The item pool was derived from the review of current clinical UE assessments and the clinical judgment of the research team (one physical therapist and two occupational therapists, each with more than 10 yr of stroke and measurement expertise). Different dimensions of UE motor function were identified for this instrument. A 4-point scale was chosen to increase the precision of the assessment. To establish content validity, we asked four additional experts (physical therapists with 4–23 yr of experience in stroke rehabilitation) to review and examine the items selected for this assessment for their relevance to the construct of UE motor function and to ensure that the instrument covers all aspects of the construct. Changes and improvements in task descriptions, instructions, and scoring criteria were then made after the review. In Phase 2, test–retest reliability and construct validity were examined. Construct validity was examined using the correlations between the ArmCAM and other clinical measures related to UE motor function after stroke.

The following principles were prioritized in developing the ArmCAM (see also the Supplemental Appendix, available online with this article at https://research.aota.org/ajot): (1) represents clinically meaningful movements that relate to UE motor function; (2) can be undertaken with a computer, tablet, or phone with a webcam; (3) utilizes objects available in the home; and (4) has distinct steps in scoring that can be easily discerned over a webcam. With this in mind, we chose 10 motor tasks that represented gross movements (two items), supported reach (one item), functional reach-and-grasp movements (three items), functional wrist movement (one item), and fine motor skills (three items). Item 1, putting the hand on top of the head, and Item 2, holding a magazine between the upper arm and side of the body, were chosen to characterize UE gross movements. Item 3, sliding a towel, was used to assess the supported reach movement. Item 4, pouring; Item 5, opening a jar; and Item 6, grasping and lifting a can to eye level, were designed to examine functional reach-and-grasp movements. Item 7, grasping and inverting a soup can, was used to check functional wrist movement. Finally, Item 8, stacking coins; Item 9, manipulating coins; and Item 10, finger opposition, were used to evaluate the fine motor skills of the hand. The research team had extensive experience in clinical measures of UE motor impairment and function (e.g., FM–UE, ARAT). Using this knowledge and experience, they selected items representing a wide range of functionally relevant UE movements to assess body functions and activities according to the ICF. These items are ordered to minimize transitions for adjusting the webcam and repositioning of the participant. Assessment objects included one magazine, one paper towel, two cups, one jar with a lid, one soup can, and three coins. Each item was given an ordinal score (0 = can perform no part of the task; 1 = performs task partially; 2 = completes the task but slowly, with difficulty, or using compensatory strategies; and 3 = performs task normally). Scored patient videos of the ArmCAM are available at https://youtu.be/aKWtpJeAAg4.

Participants

Participants were recruited from a volunteer database from May to August 2020. The participants were mostly drawn from the Vancouver metropolitan area, with approximately 10% of the participants drawn from other provinces in Canada because the study was conducted remotely by means of videoconferencing. The inclusion criteria were as follows: Participants must (1) be ≥19 yr old; (2) be >6 mo poststroke; (3) be able to follow instructions and communicate; (4) have access to the internet and a computer, tablet, or phone with a webcam on their own or with the assistance of a family member or caregiver; and (5) live in the community. The exclusion criteria were as follows: (1) neurological conditions other than stroke and (2) severe UE pain. The study protocol (Nos. H18–03101 and H13–01301) was approved by the university ethics board. Informed consent was obtained from all participants.

Procedure

Two assessment sessions were performed on different days within 1 wk using videoconferencing software (Zoom Video) to assess the test–retest reliability. Data regarding self-report use (Rating of Everyday Arm-use in the Community and Home [REACH] scale; Simpson et al., 2013) and self-perceived impact of stroke (the hand function domain of the Stroke Impact Scale [SIS–Hand]; Duncan et al., 2003) were collected at the first session, whereas clinical measures of UE were used to gather data on impairment (FM–UE; See et al., 2013) and function (ARAT; Van der Lee et al., 2001) taken from a subset (n = 10) of the participants obtained from the database of our previous studies within 1 to 8 mo before this study. The FM–UE and ARAT scores were obtained during an in-person assessment session when the subset of the participants participated in our previous study. The REACH scale was chosen because a previous study has shown that it correlated well with UE motor impairment as measured by the Chedoke–McMaster Stroke Assessment and UE function as measured by the ARAT (Simpson et al., 2013). The SIS is a reliable and valid self-report measure for people with stroke (Duncan et al., 2003). People with stroke rated how difficult it was to use their affected hand in a range of activities in the past 2 wk (e.g., turn a doorknob, tie a shoelace), which is related to their perceived hand motor function. The ArmCAM was administered at both sessions. Standardized filming positions, including camera angles and participant seating, are described in Table 1. The second session was video recorded on Zoom. One rater (Rater A, Chieh-ling Yang) administered all the tests in both sessions, and another rater (Rater B, Lisa A. Simpson) rated the ArmCAM scores by video observation. The ArmCAM scores between two sessions from Rater A were used to establish the test–retest reliability. The ArmCAM scores of the second session from Rater A and Rater B were used to establish the interrater reliability. Both raters are occupational therapists with experience in stroke rehabilitation and the developers of the ArmCAM. They were trained to administer the test properly (with 4 hr of training, including reviewing materials and discussing scoring criteria with each other).

Data Analysis

The ArmCAM total score was analyzed (sum of the 10 tests = 30). Summing up the ordinal scores from all items to create an approximately continuous variable is commonly used in the rehabilitation field. Reliability was calculated on the basis of a single measurement and absolute agreement using a two-way mixed-effects model, intraclass correlation coefficient (ICC[3,1]) for test–retest reliability, and a two-way random-effects model (ICC[2,1]) for interrater reliability. Reliability was considered excellent if the ICCs were greater than .90 and good if they were between .75 and .50 (Koo & Li, 2016). Cohen’s κ was used to analyze each individual item’s test–retest reliability and interrater level of agreement. Cutoff points for interpreting Cohen’s κ were defined in the following ranges according to Landis and Koch (1977): <0 (no agreement), 0.10–0.20 (slight agreement), 0.21–0.40 (fair agreement), 0.41–0.60 (moderate agreement), 0.61–0.80 (substantial agreement), and 0.81–0.99 (almost perfect agreement). We also calculated the standard error of measurement (SEM; Haley & Fragala-Pinkham, 2006) and minimal detectable change based on a 95% confidence interval (MDC₉₅; Beckerman et al., 2001). An MDC₉₅ percentage ([MDC₉₅/mean] × 100%) of <30% was considered as acceptable, and <10% was considered as excellent (Smidt et al., 2002). The Bland–Altman method was used to visualize the difference and mean score of each pair of measurements (Bland & Altman, 1986).

For validity, Pearson correlation coefficients were used to examine validity between the ArmCAM and SIS–Hand. Because the REACH scale is an ordinal scale, the Spearman rank correlation coefficient was used to examine the relationship between the ArmCAM and the REACH scale for all participants. The Spearman rank correlation coefficient was also used to examine the relationships of the ArmCAM with the FM–UE and ARAT for the subgroup because of the small sample size (n = 10). Correlations were considered good to excellent if they were greater than .75 (Portney & Watkins, 2007). The level of significance was set at p < .05. SPSS (Version 27.0) was used.