Effectiveness of Interventions to Address Cognitive Impairments and Improve Occupational Performance After Traumatic Brain Injury: A Systematic Review

Search Strategy

The inclusion criteria for this review were as follows: Studies were published in peer-reviewed scientific literature between May 2008 and January 2014, articles were written in English, and interventions were within the domain of occupational therapy (AOTA, 2014). For this systematic review, at least 30% of participants in each study sample were adults with TBI. Using the evidence hierarchy described by Sackett, Rosenberg, Muir Gray, Haynes, and Richardson (1996), descriptive outcome studies such as single-subject and case series designs (Level IV evidence) and case reports, narrative literature reviews, and consensus statements (Level V evidence) were included only when Level I (systematic reviews, meta-analyses, randomized controlled trials [RCTs]), Level II (two-group nonrandomized studies), or Level III (one-group, nonrandomized studies) evidence was not found. The reviews excluded qualitative studies and reports from presentations, conference proceedings, non–peer-reviewed research literature, dissertations, and theses.

The methodology consultant to the AOTA EBP Project and AOTA staff identified the search terms in consultation with the review authors and the advisory group, with the terms selected in keeping with the specific thesaurus of each database included in the search. A medical research librarian with experience in completing systematic review searches further refined the search strategies and conducted all searches. The databases and sites that were searched included Medline, PsycINFO, CINAHL, OTseeker, and the Cochrane Database of Systematic Reviews without preliminary exclusion on the basis of publication type. The review teams examined reference lists from articles that were identified for inclusion for additional potential articles, and selected journals were hand searched to ensure that all appropriate articles were included. See Supplemental Appendix 1 (available online at http://otjournal.net; navigate to this article and click on “Supplemental”) for one of the electronic search strategies for this question.

Study Selection, Data Extraction, and Risk of Bias Assessment

The EBP Project methodology consultant first eliminated references for each question on the basis of citations and abstracts. Two members of the review team evaluated each abstract, presented it to the author team, and recommended its inclusion or elimination from the review on the basis of established criteria. Disagreements were handled through consensus. We retrieved full-text versions of the articles for the remaining references and reviewed them for relevance to the question, study quality, and levels of evidence. Each included article was abstracted using an evidence table that included the level of evidence, a summary of the study methods, and findings relevant to the review question. AOTA staff and the EBP consultant reviewed the evidence tables to ensure quality control before we undertook a more in-depth review and summarization.

We assessed the risk of bias of individual studies using the methods described by Higgins, Altman, and Sterne (2011). The method for assessing the risk of bias of systematic reviews was based on work by Shea et al. (2007). Risk of bias for each study was assessed by pairs of reviewers who proposed risk characterizations to the entire author team. Disagreements were resolved through consensus.

Data Synthesis Methods

Given the heterogeneity of the included studies, we used a qualitative approach to data synthesis. We examined the studies selected for review for similarities across participants, settings, interventions, and outcomes and grouped related studies into themes.

Designations of the strength of the evidence for each theme were adapted from the system proposed by the Agency for Healthcare Research and Quality, U.S. Preventive Task Force (2012). The designation of strong evidence indicates consistent results from at least two RCTs. A designation of moderate evidence was made on the basis of one RCT or two or more studies with lower levels of evidence. The designation of limited evidence was based on few studies, flaws in available studies, or some inconsistency in the findings across individual studies. A designation of mixed evidence indicates that the findings were inconsistent across studies in a given category. Finally, a designation of insufficient evidence was used when the number and quality of studies were too limited to make any clear classification. Risk of bias appraisals were considered in strength of evidence designations.

Interventions to Address Problems With Executive Function, Attention, or Self-Awareness

We reviewed 14 studies that evaluated interventions for executive function, self-awareness, or attention, alone or in combination (6 Level I systematic reviews, 6 Level I studies, 1 Level II study, 1 Level III study). Strong evidence supports attention regulation interventions with or without goal or problem-solving training. Two Level I systematic reviews concluded that direct attention training (Cicerone et al., 2011), attention processing training, and dual-task and task shifting training (Zoccolotti et al., 2011) improved attention regulation. Four Level I RCTs concluded that evidence supports group, goal-oriented attention regulation training (Chen et al., 2011; Novakovic-Agopian et al., 2011) and dual-task training (Couillet et al., 2010; Evans, Greenfield, Wilson, & Bateman, 2009). Strong evidence supports executive function strategy training such as goal management training and metacognitive strategy instruction (2 Level I systematic reviews, Kennedy et al., 2008; Zoccolotti et al., 2011; 2 Level I RCTs, McPherson, Kayes, & Weatherall, 2009; Vas, Chapman, Cook, Elliott, & Keebler, 2011). A Level II study of goal management training had mixed results (Grant, Ponsford, & Bennett, 2012).

Strong evidence supports cognitive interventions to improve self-awareness. A Level I systematic review endorsed the benefits of feedback-related interventions (Schmidt, Lannin, Fleming, & Ownsworth, 2011), and a Level III study demonstrated pretest–posttest improvements after group treatment for self-awareness and coping (Lundqvist, Linnros, Orlenius, & Samuelsson, 2010).

Three Level I systematic reviews examined multiple intervention approaches for attention or executive dysfunction, resulting in mixed evidence. Two systematic reviews supported compensatory, behavioral, and attention remediation approaches but not computer-based stimulation interventions (Boelen, Spikman, & Fasotti, 2011; Cicerone et al., 2011). The third systematic review concluded that the evidence for compensatory, restorative, and adaptive approaches to executive dysfunction was insufficient (Chung, Pollock, Campbell, Durward, & Hagen, 2013).

Occupational performance was the primary study outcome in only one of these studies (Grant et al., 2012). Most studies used primary neuropsychological outcome metrics. In some studies, attainment scaling, self-report questionnaires, or both were used as secondary outcomes, some of which evidenced positive results in occupational performance (Couillet et al., 2010; McPherson et al., 2009; Vas et al., 2011).

Interventions to Address Problems With Memory

Twenty studies examined intervention approaches to improve memory performance after TBI (4 Level I systematic reviews, 7 Level I studies, 4 Level II studies, 5 Level III studies). Strong evidence supports general memory interventions (restorative or compensatory approaches). Two Level I systematic reviews supported general memory interventions including internal and external memory strategy training, errorless learning, and remediation approaches (Cicerone et al., 2011; Piras, Borella, Incoccia, & Carlesimo, 2011). A Level I RCT demonstrated the effectiveness of compensatory, restitution, and self-help approaches (das Nair & Lincoln, 2012).

Two studies examined the effects of interventions focused solely on memory compensation. A Level I RCT found compensatory memory strategy training to be beneficial with and without self-awareness training (Shum, Fleming, Gill, Gullo, & Strong, 2011). Level III evidence supports an ecologically oriented neurorehabilitation approach to memory compensation (Stringer & Small, 2011). Even so, a Level I meta-analysis concluded that there was insufficient evidence to support memory-related interventions (Rohling, Faust, Beverly, & Demakis, 2009). This conclusion may be explained by the fact that restorative and compensatory approaches were not differentiated in the analyses. Notably, neither das Nair and Lincoln (2012) nor Shum et al. (2011) found between-group differences in occupational performance (not measured by Stringer & Small, 2011).

Strong evidence supports the use of encoding techniques to improve recall. A Level I RCT supported systematic instruction (mastery of sequentially trained skills and “error control” techniques) over conventional trial-and-error learning for training people with TBI to use a personal digital assistant (PDA; Powell et al., 2012). Another Level I randomized crossover design study demonstrated improved recall after training in motor sequences using modeling (Zlotowitz et al., 2010). Three studies providing Level II evidence supported encoding techniques to improve recall, including self-generation (Goverover, Chiaravalloti, & DeLuca, 2010), internal memory strategy use (O’Neil-Pirozzi et al., 2010), and visual imagery (Potvin, Rouleau, Sénéchal, & Giguère, 2011). Two Level III studies supported text messaging to improve recall of therapy goals (Culley & Evans, 2010) and spaced retrieval for face–name associations (Haslam, Hodder, & Yates, 2011). Only Goverover et al. (2010) evaluated occupational performance outcomes, which were positive.

Strong evidence supports the benefits of teaching adults with TBI to use cognitive assistive technology (CAT) to compensate for memory impairments. Two Level III studies demonstrated that people with brain injury were able to learn how to use CAT (calendar software, de Joode et al., 2012; electronic aids built into a training apartment, Boman et al., 2010). A Level I systematic review supported training in PDAs and cellphones but not voice recorders or navigation devices (de Joode et al., 2010). Two Level I systematic reviews and 1 Level II study found that the NeuroPage, a programmed pager device, aided prospective memory (de Joode et al., 2010; Piras et al., 2011) and decreased caregiver burden (Teasdale et al., 2009). A Level I RCT also demonstrated beneficial effects of CAT in the form of a television-assisted prompting system (Lemoncello, Sohlberg, Fickas, & Prideaux, 2011). However, another Level I RCT did not support use of a calendar system delivered over the Internet (Bergquist et al., 2009). Finally, a Level I multicenter randomized parallel group study determined that PDAs and pencil-and-paper methods were equally beneficial (de Joode, van Heugten, Verhey, & van Boxtel, 2013). Six articles reported positive outcomes in occupational performance associated with intervention (Boman et al., 2010; de Joode et al., 2010, 2012, 2013; Lemoncello et al., 2011; Teasdale et al., 2009).

Interventions to Address Multiple Cognitive Domains Using General Compensatory Approaches

General compensatory interventions (which may involve strategy training) often simultaneously address multiple cognitive domains. Two studies provide moderate evidence to support this approach. In a Level I RCT of the Short-Term Executive Plus (STEP) approach, participants with TBI demonstrated improvement in self-reported executive function and problem solving (Cantor et al., 2014). In a Level III pilot study, combat veterans with mild cognitive impairments reported improvements in the use of compensatory strategies, increased life satisfaction, and reduced symptom severity after group-based cognitive strategy training during which they learned internal and external memory strategies and problem-solving strategies (Huckans et al., 2010). In both studies, participants appeared to make significant improvements in cognitive strategy use, but they did not appear to improve in social role or community participation.

Interventions to Address Multiple Cognitive Domains Using Computer-Based Brain Training

Two Level III studies of computer-based interventions involved small sample sizes, mixed study populations, and varied intervention frequency and intensity and rarely tied outcomes to occupational performance (Kim et al., 2009; Sullivan, Quinn, Pramuka, Sharkey, & French, 2012). As such, the evidence is insufficient to support computer-based interventions to enhance occupational performance. A retrospective chart review of military personnel with TBI found that participants reported subjective improvement in symptoms with unstructured practice using commercially available software (Sullivan et al., 2012). However, the sample was small, and the effects from the computer intervention could not be isolated relative to the concurrent traditional interventions. The other study demonstrated functional MRI brain changes for 10 people with TBI who completed supervised computerized training, but occupational performance outcomes were not evaluated (Kim et al., 2009).

Limitations

Although this systematic review used a methodology designed to minimize bias, all forms of bias could not be entirely eliminated; for example, we included only studies reported in English. Other limitations are based on the overall design and methods of individual studies, which typically used relatively small samples. In addition, although compelling evidence supports many cognitive interventions, occupational performance outcomes measurement remains less robust. Many studies used neurocognitive metrics as primary outcomes, with occupational performance as a secondary study outcome. In the nine systematic reviews examined, the impact of interventions on occupational performance was somewhat difficult to fully ascertain. Methods used to characterize occupational performance in this review relied primarily on self-report.

Interventions included in this systematic review were not evaluated for clinical feasibility. Many of the studies with the highest levels of evidence involved lengthy interventions (e.g., 108 hr of STEP provided over 12 wk; Cantor et al., 2014), which may not be replicable in all clinical settings. In advance of findings from translational research, occupational therapy practitioners should critically examine pertinent studies to inform any specific practice changes.

Implications for Occupational Therapy Research

Findings from this systematic review suggest that although many approaches to cognitive intervention yield improvements in specific cognitive domains, the short- and long-term influence on occupational performance remains less clear. This lack of clarity may be attributed to the fact that in most studies, occupational performance, if measured at all, was considered a secondary outcome of the intervention.

RCTs are needed in which occupational performance is the primary study outcome, preferably involving intervention approaches deemed beneficial in this review. Ultimately, research is also needed to expand the options for measuring occupational performance beyond that of self-report. A wider range of reliable, valid performance-based cognitive outcome measures are needed, including those with alternate forms for repeat administration. Finally, research–practice collaborations are needed to translate promising cognitive interventions described in research protocols into clinically feasible interventions.

Implications for Occupational Therapy Education

Cognitive rehabilitation remains an interdisciplinary enterprise. To contribute the much-needed perspective of occupational performance, occupational therapy practitioners must be educated, grounded, and conversant in brain–behavior relationships and the basics of human cognition, cognitive science, and cognitive psychology. Occupational therapy practitioners and students may benefit from further education in dual-task training and metacognitive strategy instruction methods.

Implications for Occupational Therapy Practice

This systematic review has several implications for practice. Occupational therapy intervention using cognitive remediation and strategy instruction approaches should be provided to adults with TBI. Cognitive intervention involving CAT has particularly strong evidence and relevance to occupational therapy practice. Occupational therapy practitioners are well positioned to lead this area of service delivery in their interdisciplinary clinical settings.

Limited research has addressed general cognitive stimulation approaches that use computers, and what little research does exist has not supported the effectiveness of these approaches. Therefore, until more research is conducted, occupational therapy practitioners should carefully consider how this intervention approach fits into the intervention plan for adults with TBI. Rather than being provided as a skilled service, cognitive stimulation using computers may be recommended as an independent leisure outlet for interested clients.