Abstract
Even after rehabilitation, more than half of people poststroke continue to have difficulties in performing everyday, functional tasks (Hartman-Maeir et al., 2007). Specifically, the most troublesome areas in chronic stroke are bathing and dressing, with 68% and 59% of people having continued difficulty, respectively, at 1 yr poststroke. People also self-report needing high levels of assistance with instrumental activities of daily living, including meal preparation (77%), housekeeping (70%), and shopping (52%), 1 yr poststroke (Hartman-Maeir et al., 2007). Despite repeated calls for occupation-based assessment and intervention, occupational therapy practitioners spend the majority of poststroke rehabilitation addressing impairments (American Occupational Therapy Association, 2014; Richards et al., 2005).
Virtual reality (VR) interventions are an example of impairment-based motor rehabilitation interventions that have been shown to be effective at alleviating the targeted impairment, such as upper extremity motor performance, but to have little effect on functional outcomes (Van Peppen et al., 2004). A gap in the stroke literature exists regarding an evidence-based intervention for addressing meaningful activity performance in a manner that allows for transfer and generalization to activities and environments.
Metacognitive strategy training (MCST) interventions, a class of interventions that use cognitive strategy use to overcome performance deficits, are top–down and seek to elicit change directly on functional outcomes (Polatajko et al., 2012). Cognitive strategies may be used to aid in learning or performance and allow for planning, monitoring, and adapting of actions (Toglia et al., 2012). MCST interventions have building evidence for positive outcomes on both trained and untrained tasks within the stroke population (McEwen et al., 2009, 2010, 2015).
In this study, we sought to further improve on the positive effects of MCST by pairing it with a VR rehabilitation intervention for poststroke hemiparesis. We have developed an intervention, MetacogVR, that consists of a bottom-up, VR motor rehabilitation component and a top-down, cognitive strategy use practiced within client-chosen goals in people with stroke. The rationale is that direct improvements in motor impairment when drawn on during task performance and guided by cognitive strategy use may lead to greater functional performance than either approach in isolation.
Flexible Action Articulated Skeleton Toolkit software (University of Southern California, Institute for Creative Technologies, Los Angeles)—a VR process that uses Microsoft Kinect—(Microsoft Corp., Redmond, WA; Boone et al., 2017)—may be used in conjunction with free, online flash games to create a low-cost, motivating environment for achieving hundreds of challenging movement repetitions (Lauterbach et al., 2013; Suma et al., 2011). Within this process, specified movements (e.g., elbow flexion) that meet a set threshold (e.g., 50°) may be used to create a corresponding event within games tailored to client interests. Through the addition of task-based practice of client-chosen goals, MetacogVR integrates impairment and function through the application of three key active ingredients: movement repetitions, guided discovery, and cognitive strategy use. The MetacogVR approach is innovative because it draws on the strengths of impairment-based approaches for remediation of upper extremity impairment and the strengths of cognitive strategy use for achieving improvements in activity performance; pairing of the two approaches was hypothesized to have both summative and interactive effects.
To our knowledge, no previous study has explored the combination of such approaches. Guidelines for developing and testing multi-ingredient interventions, including those combining previously tested methods, call for feasibility testing to explore outcomes similar to that proposed in this study (Craig et al., 2008). Exploring feasibility components early in the intervention development process will allow for identification of strengths and weaknesses of the intervention, which, in turn, will allow for modification and strengthening of future studies.
The purpose of this feasibility study was threefold: (1) to determine acceptability of the intervention (satisfaction and motivation); (2) to determine the recruitment rate and retention rate of participants; and (3) to evaluate the preliminary effect of MetacogVR on measures of upper extremity motor performance, cognition, self-efficacy, and occupational performance for trained and untrained tasks as well as participation to determine what measures best captured the effect of MetacogVR (Bowen et al., 2009; Craig et al., 2008; Lancaster et al., 2004).
Method
This study had a single-group, pre–post design. A subset of participants (n = 5) also completed questionnaires to gauge level of satisfaction and to measure motivation levels midway through the intervention. The single-group design is appropriate during feasibility testing of an intervention because the study was not seeking to evaluate intervention efficacy but solely a preliminary effect on measures to inform outcome measurement selection for future studies (Lancaster et al., 2004). This study was approved by the University of Missouri institutional review board.
Participants
Participants were recruited through community support groups and through the Stroke Management and Rehabilitation Team (SMART) database at Washington University in St. Louis. Participants provided written informed consent. People met the following inclusion criteria: (1) >6 mo poststroke; (2) ages 18–75 yr; (3) no ongoing occupational therapy services; (4) self-reported unmet functional goals; (5) upper extremity hemiparesis with a score of 1–3 on the National Institutes of Health (NIH) Stroke Scale (NIHSS; Brott et al., 1989) arm motor item; and (6) a minimum of 10° active range of motion at the shoulder, elbow, or wrist. Participants were excluded with the following criteria: (1) NIHSS aphasia score of ≥2, (2) cognitive impairment with a score of <21 on the Montreal Cognitive Assessment (Nasreddine et al., 2005), (3) additional neurological diagnoses, (4) severe depressive symptoms with a score of >21 on the Patient Health Questionnaire (Kroenke et al., 2001), (5) visual acuity of <20/40, or (6) spasticity of >3 on the Modified Ashworth Scale (Bohannon & Smith, 1987).
Data Collection
Recruitment and retention data were collected. Assessments were conducted pre- and post-intervention and at 3-mo follow-up by the treating occupational therapy practitioner with 5 yr experience (Table 1). To measure client satisfaction and motivation levels, we used the Client Satisfaction Questionnaire–8 (CSQ–8; Attkisson & Greenfield, 1994) and Intrinsic Motivation Inventory (IMI; Ryan et al., 1990).
Outcome Measures Evaluating Effect
Note. ADLs = activities of daily living; COPM = Canadian Occupational Performance Measure; Neuro-QoL = Quality of Life in Neurological Disorders; NIH = National Institutes of Health; PQRS = Performance Quality Rating Scale; PROMIS = Patient-Reported Outcomes Measurement Information System; PS–SES = Participation Strategies Self-Efficacy Scale; WCPA = Weekly Calendar Planning Activity.
Intervention
MetacogVR is a 12-wk intervention with three 1-hr sessions per week. The intervention was administered in a university research laboratory (Figure 1). The treating occupational therapy practitioner led all sessions. First, the client identified goals with the Canadian Occupational Performance Measure (COPM; Law et al., 2014). Five of the six goals were trained in the intervention, and one was used as a method of assessing generalization of skills. Then, the intervention began an alternating format of two weekly, 1-hr VR sessions and one weekly, 1-hr, task-based session. Client-tailored VR sessions primarily focused on achieving a high volume of relevant movement repetitions while playing three to five games each session. Each session began with a discussion of previously assigned homework and ended with a “so what?” discussion of how improvements in the targeted movements assist functionally and how clients could further incorporate the movements functionally. Fatigue and pain were monitored multiple times each session per self-report, and breaks were taken accordingly.
Intervention setup at the university research laboratory.
The third weekly session was task-based and involved practice of COPM goals with an emphasis on cognitive strategy use. Cognitive strategies, such as “slowing down” or altering one’s body position, can be used to allow the client to problem solve and overcome performance deficits. Use of guided discovery permits the occupational therapy practitioner to facilitate learning and cognitive strategy use through methods such as probing questioning or modeling as opposed to the typical directive role of an occupational therapy practitioner (Henshaw et al., 2011).
Analysis
Data were analyzed with IBM SPSS Statistics (Version 24; IBM Corporation, Armonk, NY). First, recruitment rate and retention rate were calculated. CSQ–8 and IMI data were analyzed with descriptive statistics. All outcome data were checked for outliers and for normal distributions of differences with the Shapiro–Wilks test. Data were normally distributed; therefore, parametric procedures were used. Mean changes, standard deviations (SDs) , and Cohen’s d effect sizes were calculated. Cohen’s d effect sizes were interpreted as follows: 0.2 = small, 0.5 = medium, and 0.8 = large (Cohen, 1992).
Results
Acceptability
Per the IMI, there was an average participant rating of 6.0 (SD = 0.9) that is indicative of a high level of motivation. Results suggest a high level of satisfaction with the intervention as indicated by an average score of 28.8 (SD = 2.1) out of a possible 32 on the CSQ–8.
Recruitment and Retention
Over a 10-mo period, 81 participants underwent initial screening. Of the 30 who met initial screening criteria, 27 were fully screened, and 12 were eligible. Participants who did not qualify were too high or low functioning in motor performance, had other neurological diagnoses, severe depressive symptoms, or severe aphasia. Two participants dropped out after screening, citing they were too busy to proceed, leading to a recruitment rate of 15%. There was a high retention rate of 100%. The sample on average was about age 60 yr and 5 yr poststroke, with hemiparesis mainly affecting the prestroke dominant side (Table 2).
Participant Demographics (N = 10)
Note. GED = General Educational Development; HS = high school; M = mean; SD = standard deviation.
Outcomes
Means, SDs, and Cohen’s d effect sizes are presented in Table 3. At 3-mo follow-up, the largest effect was demonstrated on upper extremity motor performance via the Fugl–Meyer Assessment (d = 3.3). Large effects at follow-up were also demonstrated on subjective and objective measures of occupational performance via the Performance Quality Rating Scale (PQRS; Martini et al., 2015) for trained goals (d = 0.9) and COPM performance scores for trained and untrained goals (ds = 1.6 and 1.1, respectively). We found a large effect for participant satisfaction with occupational performance via the COPM for trained and untrained goals (ds = 1.1 and 0.9, respectively), for the participation outcome of PROMIS (Patient-Reported Outcomes Measurement Information System; Cella et al., 2010)–Ability to Participate in Social Roles and Activities (d = 0.8), and for the number of accurate appointments entered on the Weekly Calendar Planning Activity (Lahav et al., 2018; Toglia & Berg, 2013; d = 1.2). A medium effect was demonstrated on cognitive impairment measured with NIH Toolbox Flanker (Weintraub et al., 2013) at follow-up (d = 0.5). The Participation Strategies Self-Efficacy Scale (Lee et al., 2018) and the NIH Toolbox Card Sort (mental flexibility and attention; Weintraub et al., 2013) demonstrated negligible effects at 3-mo follow-up.
Results of Outcome Assessments
Note. CI = confidence interval; COPM(P) = Canadian Occupational Performance Measure performance score; COPM(S) = Canadian Occupational Performance Measure satisfaction score; M = mean; Neuro-QoL = Quality of Life in Neurological Disorders; NIH = National Institutes of Health; PQRS = Performance Quality Rating Scale; PROMIS = Patient-Reported Outcomes Measurement Information System; PS–SES = Participation Strategies Self-Efficacy Scale; SD = standard deviation; UE = upper extremity; WCPA = Weekly Calendar Planning Activity.
Effect size interpretation: 0.2 = small, 0.5 = medium, 0.8 = large.
Mean change from preintervention to 3-mo follow-up exceeds the minimal detectable difference of 1.7 for the COPM(P) or the minimally important difference of 5.25 for the Fugl–Meyer (Cup et al., 2003; Page et al., 2012).
Discussion
The purpose of this feasibility investigation was to determine acceptability, recruitment and retention rates, and the preliminary effect of MetacogVR in people with chronic stroke across a variety of outcome measures, including occupational performance outcomes on trained and untrained tasks. Collectively, the findings will inform recruitment expectations and outcome measure selection for future evaluation of the MetacogVR intervention. MetacogVR was acceptable to key stakeholders as evidenced by high levels of motivation and satisfaction. Recruitment within research, including stroke rehabilitation research, has been a consistent barrier, as the recruitment rate of 15% in the current study reflects. For example, within the Extremity Constraint-Induced Therapy Evaluation trial, only 6.5% of people contacted and initially screened were enrolled (Blanton et al., 2006).
Although a recruitment rate of 15% is lower than desired, strategies to increase recruitment and to maintain the high retention levels will allow for future studies to have adequate sample sizes. For example, using a multifaceted recruitment approach to also include provider referrals or social media advertising may enhance recruitment efforts. Although 100% retention is not likely possible in a larger scale evaluation of this intervention, we can expect a high retention rate within an acceptable range in future studies.
Results support the use of the Fugl–Meyer Assessment, COPM, PQRS, and PROMIS–Ability to Participate in Social Roles and Activities as measures in future studies evaluating the effect of MetacogVR. Large effects on these measures in the current study indicate that they are sensitive to positive changes elicited by MetacogVR on upper extremity motor performance, participation, and objective and subjective perceptions of client-chosen goal performance. However, in future studies, researchers should use a measure of self-efficacy that is more specific to self-efficacy within occupational performance. Although the eight-item PROMIS–Ability to Participate in Social Roles and Activities assessment captured the expected increase in participation at the 3-mo follow-up, a more in-depth assessment of daily life participation may provide a fuller picture as to how participation is changing in future studies.
Task-specific training acknowledges the necessity to approach motor impairment through the use of functional movements within the contexts of a task that is of meaning to the client and through repetitions of that task (Hubbard et al., 2009). A review of therapy interventions based on these task-specific training principles concluded that the effects of such interventions were exclusive to trained activities (Veerbeek et al., 2014). The focus of MetacogVR after learning cognitive strategies is advantageous because it allows for transfer and generalization of learning. Transfer and generalization are critical to poststroke rehabilitation because it is impractical to address all deficits within the limited amount of time allotted in rehabilitation. Here, preliminary data are presented for the ability of MetacogVR to elicit improvements on both trained and untrained client-chosen goals. A key finding is that the sample demonstrated no effect on the participation measure postintervention; however, at 3-mo follow-up, the participants demonstrated a large effect. This finding further underscores the idea that the skills learned were being transferred in a meaningful manner.
Limitations and Future Considerations
Although a small sample and pre–post design were sufficient because of the exploratory nature of this study, generalizability of these findings at this time is limited (Bowen et al., 2009). A threat to internal validity is the lack of a control group or a treatment comparison group. Without a control group, it is more difficult to fully attribute the observed treatment effect to MetacogVR. However, it is well documented that poststroke functional deficits are relatively stable long-term in the absence of intervention (Meyer et al., 2015). Moreover, the same occupational therapy practitioner administered the intervention and outcome assessments, possibly leading to a bias in performance results; however, it does not appear systematic as evidenced by a lack of improvement on some measures at postintervention and a decrease in scores on some measures at follow-up.
Despite these limitations, the study design was adequate and practical for achieving the purposes of evaluating feasibility outcomes. The results from the current feasibility study provide the groundwork for future efficacy studies. A plausible design for future efficacy studies would be a double-blind randomized controlled trial with the following three groups: a MetacogVR group, a VR group, and an MCST group. Inclusion of a longer follow-up period would allow for knowledge of maintenance of effects over time.
Implications for Occupational Therapy Practice
The results of this study have the following implications for occupational therapy practice:
MetacogVR is a feasible intervention combining cognitive strategy use and VR motor rehabilitation to overcome the historical lack of transfer within impairment-based interventions and to further enhance improvements in function resulting from cognitive strategy use.
The effect of MetacogVR is best captured with measures of upper extremity function, objective and subjective measures of occupational performance, and participation measures.
Conclusion
Preliminary results indicate that the MetacogVR intervention is highly acceptable to the target population; the recruitment rate is on par with other rehabilitation studies; and the Fugl–Meyer Assessment, COPM, PQRS, and PROMIS–Ability to Participate in Social Roles and Activities are appropriate measures for use in future studies. The findings of this feasibility study provide support for future investigations of the efficacy of MetacogVR.
Footnotes
Acknowledgments
This research was conducted at the Program in Occupational Therapy, School of Medicine, Washington University in St. Louis.