Abstract
This secondary analysis quantified the psychometric properties of the Ohio Modified Arm–Motor Ability Test (OMAAT) in a sample of neurologically stable chronic stroke survivors (n = 67, 40 men; mean age 59.8 yr, standard deviation = 12.8; 42 White, 23 Black, 2 other; 92.5% right-sided lesion; 44 ischemic stroke). Findings indicate high OMAAT internal consistency (Cronbach’s α = .97, ordinal α = .98, Gugiu’s bootstrap reliability = .97), unidimensionality, and strong positive factor loadings for all 20 OMAAT items. Convergent validity between OMAAT and Action Research Arm Test total scores was strong (r = .90, p < .0001). The OMAAT is the first short measure of upper extremity functional limitation available to clinicians and researchers that includes an administration manual and that has been examined using nonparametric psychometrics. A detailed administration manual is provided as a supplement to this article.
Upper extremity (UE) hemiparesis remains a commonly encountered motor impairment after a variety of neurological and musculoskeletal disorders. When these debilitating conditions occur, measurement of clients’ UE functional limitation levels is critical to designing cost-effective, appropriate interventions and to determining whether clients are increasing their ability to perform activities likely to be carried out in the community.
The Arm Motor Ability Test (AMAT; Kopp et al., 1997) is often used to measure UE functional limitation in rehabilitative trials enrolling stroke survivors (e.g., Levy et al., 2016; Page, Levin, Hermann, Dunning, & Levine, 2012; Singer, Vallence, Cleary, Cooper, & Loftus, 2013) and could likely be applied to other conditions with paretic UEs. It requires clients to perform 13 common unilateral and bilateral UE tasks. Performance on each task is timed by the evaluator, rated by the evaluator using a 6-point functional ability scale (ranging from 0 [does not perform with affected arm] to 5 [does use arm at a level comparable to unaffected side]) that examines paretic UE use, and rated by the evaluator using a 6-point quality of movement scale (ranging from 0 [no movement initiated] to 5 [normal movement]). In addition, the evaluator must also time and rate performance for AMAT task components (e.g., picking up and using eating utensils). In total, the AMAT includes 13 compound tasks consisting of 28 task components. Consequently, when performed in accord with its original instructions (Kopp et al., 1997), AMAT administration can be time (45 min) and labor intensive, creating barriers to its routine clinical implementation.
In light of these limitations, our overall goal was to develop a psychometrically robust, time-efficient clinical measure that assessed performance of real-world tasks and could be used across diagnoses. As a first step, this article reports the modification, optimization, and psychometric evaluation of the AMAT in a cohort of neurologically stable chronic stroke survivors. The resulting Ohio Modified Arm–Motor Ability Test (OMAAT) modifies the AMAT by reducing the number of tasks to 9 and task components to 20 and using a single rating scale that evaluates ability to carry out functional activities. To optimize ease of implementation, Supplemental Appendix A (available online at http://otjournal.net; navigate to this article, and click on “Supplemental”) provides an administration manual with detailed written instructions and pictures depicting performance of each OMAAT item. Total administration time is approximately 30 min. Whereas previous studies (e.g., Chae, Labatia, & Yang, 2003; Kopp et al., 1997; O’Dell, Kim, Finnen, & Polistena, 2011) have compared the AMAT to general UE measures that focus minimally on distal function, psychometric evaluation of the OMAAT is enhanced by comparisons to measures of distal UE function. Additionally, we used analytic methods designed for instruments collecting ordinal-level data.
Method
Participants
Participants were recruited for the trial from across the Midwestern United States using active and passive recruiting techniques (Page & Persch, 2013). For example, we placed print advertisements in clinics near enrolling sites, provided in-services to local clinicians, and communicated with local stroke support groups. Potential participants were screened using the following eight eligibility criteria: (1) ≥10° of active flexion in the paretic wrist and at least two fingers of the paretic hand; (2) experienced a single stroke ≥12 mo before study enrollment; (3) Mini-Mental State Exam (Folstein, Folstein, & McHugh, 1975) score ≥24; (4) age ≥18 and ≤75 yr; (5) no excessive spasticity in the paretic UE (Modified Ashworth Scale [Bohannon & Smith, 1987] score ≥2 at the paretic elbow, wrist, or digits); (6) no significant pain in the more affected UE (score ≥5 on a 10-point visual analog scale); (7) not receiving any motor rehabilitation through clinical care or research; and (8) no other conditions that, in the opinion of the investigators, may hamper ability to participate in the trial.
Study Design
The current study is a secondary analysis of data obtained from a cohort of stroke survivors enrolled in an ongoing Phase IIb trial. The OMAAT was administered in the same fashion and by the same rater across participants as part of a larger battery of outcome measures. The current study used data collected only before any interventions took place.
Testing and Instruments
Several measures had been coadministered with the OMAAT as part of the larger trial, some of which were included in this analysis to determine internal consistency and convergent validity. Two licensed therapists, acting as blinded raters, were certified and recertified on the measures every 3 mo using standardized live and video-based interrater reliability checks. Raters 1 and 2 had used outcome measures consistently for 10 and 3 yr, respectively. Both raters possessed more than a decade of experience as a licensed therapist.
In addition to the OMAAT, measures administered about 1 wk after screening and consenting were the AMAT and the Action Research Arm Test (ARAT; Lyle, 1981). The AMAT is a 13-item test in which performance on various common UE activities (e.g., donning a shirt, cutting meat with a fork and knife) is timed and rated using previously described functional ability and quality of movement scales. The ARAT is a laboratory-based measure of paretic UE functional limitation. It is a 19-item test divided into four categories (grasp, grip, pinch, and gross movement), with 16 of the 19 ARAT items measuring distal regions of the UE (e.g., pinching a ball bearing between the thumb and each digit of the affected hand), making it an ideal comparator to the OMAAT. Each ARAT item is graded on a 4-point ordinal scale (ranging from 0 [can perform no part of the test] to 3 [performs test normally]) for a total possible score of 57. The ARAT has high intrarater (r = .99) and retest (r = .98) reliability and validity (Hsieh, Hsueh, Chiang, & Lin, 1998; Van der Lee et al., 2001), all in stroke-induced hemiparesis.
Data Analyses
Participant demographics were summarized using descriptive statistics. Convergent validity was assessed by computing the Pearson correlation between the OMAAT and the ARAT. Cronbach’s α was used as the traditional reliability statistic for the OMAAT (Cronbach, 1951; Nunnally & Bernstein, 1994). However, Cronbach’s α may underestimate reliability because it is designed for continuous data. Therefore, ordinal α was used to provide an appropriate, nonparametric measure of internal consistency (Gadermann, Guhn, & Zumbo, 2012; Gugiu, Coryn, & Applegate, 2010). Finally, Gugiu’s nonparametric bootstrap reliability (Dembe, Lynch, Gugiu, & Jackson, 2014), which is appropriate for nonnormal data, was used to confirm these findings. We chose to use multiple methods to estimate reliability to ensure that findings were consistent.
The OMAAT’s dimensionality was evaluated using latent parallel analysis (LPA) and ordinal exploratory factor analysis (OEFA). Latent parallel analysis was conducted to determine the number of factors to retain (Hayton, Allen, & Scarpello, 2004; O’Connor, 2000) and is appropriate for ordinal data that are continuous and normally distributed at the latent level (Gugiu et al., 2010; Gugiu, Coryn, Clark, & Kuehn, 2009; Stout, 1987; Timmerman & Lorenzo-Seva, 2011). In other words, the ordinal observed-score data generated by the OMAAT are appropriate for LPA because upper extremity motor abilities are normal and continuously distributed on the latent level. In recent years, Monte Carlo–based LPA has become the preferred method for factor extraction because it is more accurate than other methods such as the Kaiser criterion and the scree plot (Garrido, Abad, & Ponsoda, 2013; Hayton et al., 2004; O’Connor, 2000).
The polychoric correlation (Drasgow, 2004; Jöreskog, 1994) matrix of OMAAT data, generated as part of the LPA procedure, was used as input for OEFA (Bartholomew & Knott, 1999; Muthén & Muthén, 2012), which is the appropriate factor analytical technique for these data. The number of factors to retain (Hayton et al., 2004) and method of rotation (Osborne & Costello, 2009) were also defined using the results of LPA. Promax rotation (Hendrickson & White, 1964) was reserved for multifactor solutions. Items that failed to load substantially (<0.30) were removed from the dataset.
Missing data were handled using multiple imputation, which is preferred to case deletion (i.e., listwise or pairwise), because the latter would reduce sample size and introduce sampling bias (Enders & Bandalos, 2001). Although the mean or a single regression imputation value is often used in place of missing data, these approaches are known to underestimate variance and overestimate correlations (Brown, 2014). Accordingly, multiple imputation using the fully conditional specification was chosen to impute OMAAT and ARAT data (White, Royston, & Wood, 2011). This method, also known as multiple imputation using chained equations (MICE), overcomes the limitations of imputation using the mean or single regression imputation value and is appropriate when data are ordered categorically.
After 10 burn-in iterations, 5 imputations (Allison, 2003; Yuan, 2001) of OMAAT and ARAT data were performed using the MICE (White et al., 2011) method. The median imputed value for each missing data point was used to create the final datasets. Psychometric evaluation of the OMAAT and ARAT was performed using SAS 9.4 (SAS Institute, Cary, NC).
Results
Descriptive analyses of OMAAT data (k = 20; n = 67) revealed 27 cases of missing data, a 2.01% missing data rate (MDR). Similarly, analyses of the ARAT (k = 19; n = 67) revealed 2 cases of missing data, a 0.15% MDR. Given the low MDR, the decision was made to impute missing values.
Participant Demographics
The sample included 67 survivors of chronic stroke (40 men); mean age was 59.8 yr (standard deviation [SD] = 12.8). A total of 42 participants were White, 23 were Black, and 2 were other races; 92.5% had a right-sided lesion. Participants’ type of stroke was ischemic (n = 44), hemorrhagic (n = 15), or unknown (n = 8). Average OMAAT and ARAT total scores were 57.55 (SD = 20.12) and 31.22 (SD =16.58), respectively.
OMAAT Psychometrics
OMAAT Reliability Determination.
Multiple measures of internal consistency were obtained to provide an initial estimate of OMAAT reliability. Cronbach’s α was .97, ordinal α was .98, and Gugiu’s bootstrap reliability was .97. These estimates, as well as their 95% confidence intervals, are depicted in Table 1.
Multiple Measures of OMAAT Internal Consistency
Note. CI = confidence interval; OMAAT = Ohio Modified Arm–Motor Ability Test.
First instance reported.
Dimensionality.
Latent parallel analysis of the OMAAT was performed by
Generating a polychoric correlation matrix for OMAAT items;
Using the polychoric correlation matrix as input for LPA;
Conducting Monte Carlo simulations to create 250 random datasets designed to match the number of respondents (n = 67) and distributional characteristics (i.e., mean, variance, skewness, kurtosis) of OMAAT items; and
Computing, extracting, and comparing eigenvalues for the real and random datasets.
Using this procedure, we found that LPA suggested a single-factor (i.e., unidimensional) solution. Ordinal exploratory factor analysis of the OMAAT was conducted based on these data, without rotation. This single-factor model resulted in strong, positive factor loadings for all 20 OMAAT items (Table 2).
OMAAT Factor Pattern
Note. OMAAT = Ohio Modified Arm–Motor Ability Test.
Bilateral task.
120-s limit.
Convergent Validity.
Computation of total scores is justified when scales are unidimensional. Therefore, convergent validity was evaluated by determining the Pearson correlation between OMAAT and ARAT total scores. The association between these two scales was strong (r = .90, p < .0001).
Discussion
Previous studies examining the AMAT’s psychometric values applied single, conventional psychometric methods that do not compute its reliability and dimensionality as an ordinal-based measure and estimated its convergent validity using measures of UE impairment that only sparsely focus on distal UE movement (Chae et al., 2003; Kopp et al., 1997), although the AMAT is a distally based measure of functional limitation. In contrast, we applied a variety of rigorous, appropriate methods to examine the OMAAT’s measurement properties and confirm its status as measuring a single construct. According to corroborating results of these analyses, the OMAAT appears to boast strong measurement properties and to be appropriate for clinical use as a stand-alone UE functional scale. This conclusion is supported by high reliability values using several different methods (all ≥.97; see Table 1), the OMAAT’s dimensionality as a single construct (again using several different methods; see Table 2), and its high convergent validity with an appropriate comparator of UE functional limitation (i.e., the ARAT; r = .90).
Moreover, whereas previous AMAT analyses enrolled only about 30 participants (Chae et al., 2003; Kopp et al., 1997), our study enrolled more than twice as many participants and used Monte Carlo simulations to guide factor retention decisions. This aspect of our study constitutes a significant improvement over previous work because samples that contain fewer than 50 participants may contribute, in part, to overestimating reliability (Javali & Gudaganavar, 2011). In addition, our use of Monte Carlo simulations to inform factor retention decisions increases the probability that the finding of OMAAT unidimensionality is accurate (Hayton et al., 2004; O’Connor, 2000).
Regarding stroke, paretic UE rehabilitation efforts stand to benefit from use of standardized OMAAT procedures. Such standardization is important given the variety of UE pathologies and patient heterogeneity that can obfuscate poststroke UE testing (Duncan, Lai, & Keighley, 2000). Despite these needs, neither the original AMAT report (Kopp et al., 1997) nor subsequent efforts (Chae et al., 2003; O’Dell et al., 2011) offered detailed procedures on how to administer its items. Although the StrokEDGE Taskforce (2011) provided instructions for administering the AMAT, this resource lacked detailed information on required materials and pictures of item administration. To address this need, Supplemental Appendix A serves as a reference tool that can be used to administer the OMAAT. The psychometrics and instructions for stroke-specific measures of UE impairment (Sullivan et al., 2011) and functional limitation in laboratory-based tasks (Yozbatiran, Der-Yeghiaian, & Cramer, 2008) have been previously reported. However, to our knowledge, this article is the first to articulate the psychometrics and instructions for a functionally based UE measure that has high potential for clinical use.
Study Limitations
Despite the improvements offered by the OMAAT, it is important to acknowledge several limitations and that additional research is needed. First, although the current study provides data regarding OMAAT unidimensionality, internal consistency, and convergent validity, additional psychometric analyses (e.g., test–retest reliability, floor and ceiling effects, responsiveness) are needed before the OMAAT can be considered a robust measure of UE functional limitation. Second, future psychometric evaluation of the OMAAT should use larger samples of stroke survivors (e.g., N > 100), which would enable direct comparisons with recent AMAT responsiveness data (Fulk, Martin, & Page, 2016). Third, future studies should include empirical evaluation of OMAAT clinical utility, including evaluation in clinical and research settings. Finally, it is important to evaluate the OMAAT in a variety of stroke survivor populations (e.g., acute and chronic; mild, moderate, and severe impairment).
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
The OMAAT exhibits high internal reliability and validity in measuring the construct of poststroke UE functional limitation.
The OMAAT is a short measure that can be administered in clinical settings to ascertain UE functional limitation.
Conclusion
UE functional limitation measures are important to discern clients’ potential for reintegration into the community after rehabilitation discharge and their likely performance on activities that are important to daily life. Using a variety of rigorous and appropriate methodologies, this study found high internal consistency and strong reliability and validity associated with the OMAAT, a brief test of UE functional limitation.
Acknowledgments
This study is registered with ClinicalTrials.gov (NCT01651533) and was funded, in part, by National Institutes of Health/National Center for Complementary and Alternative Medicine Grant 7R01AT004454-03.
Supplemental Material
Supplementary material for Ohio Modified Arm–Motor Ability Test (OMAAT): An Optimized Measure of Upper Extremity Functional Limitation in Hemiparetic Stroke
Supplementary material, sj-pdf-1-aot-10.5014_ajot.2018.025445.pdf for Ohio Modified Arm–Motor Ability Test (OMAAT): An Optimized Measure of Upper Extremity Functional Limitation in Hemiparetic Stroke by Andrew C. Persch, Alexis Wagner, Mallory Fleming, P. Cristian Gugiu and Stephen J. Page in The American Journal of Occupational Therapy
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
