Abstract

Linked article:
BACKGROUND
Incidence and Causes of Traumatic Brain Injury
Traumatic brain injury (TBI) occurs when there has been an alteration to brain function (e.g., loss of conscious, memory loss, neurologic defects) due to an external force, such as the impact of an object to the head, blast waves from an explosion, swift acceleration or deceleration, or the penetration of a foreign object into the brain (Maas, Stocchetti, & Bullock, 2008; Menon, Schwab, Wright, & Maas, 2010). TBI is a major concern worldwide and affects 10 million people annually (Hyder, Wunderlich, Puvanachandra, Gururaj, & Kobsingye, 2007). While, traditionally, road traffic injuries are the most common cause of TBI (Hyder et al., 2007; Tagliaferri, Compagnone, Korsic, Servadei, & Kraus, 2006), an increasing number of military personnel and civilians are acquiring TBI as a result of blast injuries (Fear et al., 2009; Hyder et al., 2007; Warden, 2006). This is due to recent combat operations and unrest in the Middle East. Other causes of TBI include falls, violence, and other unintentional injuries (Hyder et al., 2007). With increasing numbers of people with TBI globally, and the substantial impact on individuals with the injury, TBI is a concern for all layers of society.
Severity and Sequelae
Levels of injury range from mild to severe, determined by measures such as duration of coma or post-traumatic amnesia (PTA), Glasgow Coma Scale (GCS) scores and similar measures, and the nature and extent of functional impairments following the injury. Individuals with TBI experience various combinations of physical, cognitive, sensory, and emotional symptoms, which are largely determined by the severity of the impact, the location of the impact, and whether the TBI is a closed or open injury (National Institute of Neurological Disorders and Stroke, 2002). Table 1 illustrates sequelae commonly associated with TBI.
Common Sequelae of TBI
Typically, increased severity of injury is related to degree and extent of impairment post-injury. However, even individuals experiencing mild injuries can experience difficulties in one or more areas. In addition, most individuals who experience a TBI have a period of recovery following the injury with a return of lost functioning. However, in most cases (particularly those with moderate or severe injuries), the individual's recovery will “plateau” below his or her pre-injury functioning level.
Return to Work Following TBI
Individuals with TBI often experience difficulties in becoming competitively employed or maintaining employment after sustaining TBI as a result of the post-injury difficulties they exhibit (Andelic, Stevens, Sigurdardottir, Arango-Lasprilla, & Roe, 2012; Dikmen, Temkin, Machamer, Holubkov, Fraser, & Winn, 1994; Ezrachi, Ben-Yishay, Diller, & Rattock, 1991; Ruttan, Martin, Liu, Colella, & Green, 2008). Reported unemployment rates of people with TBI have ranged from 45% to 78% (Doctor, Castro, Temkin, Fraser, Machamer, & Dikmen, 2005; Kendall, Muenchberger, & Gee, 2006; Yasuda, Wehman, Targett, Cifu, & West, 2001). Variation in return to work (RTW) rates can be associated with differences in sample characteristics, methodologies, and measures of employment. Additionally, being employed has been associated with better quality of life (QOL) among those with TBI (O'Neill et al., 1998; Steadman-Pare, Colantonio, Ratcliff, Chase, & Vernich, 2001).
The injury and its effects can be overwhelming not just to individuals with TBI and their families (Cifu, Craig, & Rowland, 1996), but also for other layers of society. Individuals and families experience financial burden due to loss of jobs and wages as well as medical and rehabilitation expenses (Fadyl & McPherson, 2009), while communities and employers experience lost workdays and productivity (Andlin-Sobocki, Jonsson, Wittchen, & Olesen, 2005). TBI can also impose substantial burden to low-, middle-, and high-income countries through publicly funded healthcare and rehabilitation services, and financial support of injured persons (Hyder et al., 2007). The U.S. Center for Disease Control and Prevention (CDC) reported that medical and indirect costs of TBI (e.g., lost productivity) in 2000 were estimated at $76.5 billion in the United States (Finkelstein, Corso, & Miller, 2006), while the total annual cost in Europe for individuals with TBI was estimated at 386 billion euros (Andlin-Sobocki et al., 2005). Thus, the cost of TBI to the individual and society is staggering. However, there is evidence that rehabilitation is effective at easing this burden, namely by returning individuals with TBI to work (Doctor et al., 2005; Kendall et al., 2006; Yasuda et al., 2001).
Shames and colleagues (2007) conducted a comprehensive review of the literature regarding factors associated with successful return to work (RTW) following TBI and current rehabilitation strategies. They note that prediction of RTW is a complex interaction of variables, including pre-morbid factors (i.e., age, education, and prior employment history), injury-related factors (type, severity, and location), post-injury impairments, and personal and environment factors (e.g., marital status, alternative income sources, social support). Because of this complexity and the unique characteristics of each TBI patient, prediction of RTW for an individual is not feasible. They note that the literature includes numerous examples of high-risk patients who did achieve successful RTW if given sufficient rehabilitation and support.
Research cited by Shames et al. (2007) has generally found that the weight of evidence indicates these factors are frequently found to contribute to poor employment outcomes:
More severe injury (Kreutzer et al., 2003; Trexler, Trexler, Malec, Klyce, & Parrott, 2010; Wagner, Hammond, Sasser, & Wiercisiewski, 2002), as measured by GCS, PTA duration of rehabilitation, or other means; Higher age at injury (Keyser-Marcus et al., 2002; Kreutzer et al., 2003); Male (Cifu et al., 1999; Kreutzer et al., 2003); Lower levels of pre-injury educational and/or occupational status (Hart, Whyte, Polansky, Kersey-Matusiak, & Fidler-Sheppard, 2005; Keyser-Marcus et al., 2002; Walker, Marwitz, Kreutzer, Hart, & Novack, 2006); Less social support from friends, family, neighbors, church, etc. (Yasuda et al., 2001); Significant physical, psychosocial, or cognitive impairments (Keyser-Marcus et al., 2002; McCrimmon & Oddy, 2006; Wagner et al., 2002; Walker et al., 2006); Member of a minority group (Arango-Lasprilla et al., 2008; Hart et al., 2005; Kreutzer et al., 2003); and A history of substance abuse (Bogner, Corrigan, Mysiw, Clinchot, & Fugate, 2001; Corrigan, 1995; Wagner et al., 2002).
This list is far from exhaustive; many other factors have been studied, such as financial status and receipt of a settlement or public benefits following the injury. However, the list illustrates the complex nature of RTW and the difficulty in applying a prediction model to individual cases. In addition, the research literature is international, which brings into consideration differences across economies, cultures, health care and rehabilitation systems, and national “landscapes” of public and social disability policies.
Post-Acute RTW Interventions
Following acute care and rehabilitation, post-acute rehabilitation services focus on assisting the individual to adjust to on-going impairments and re-enter their communities, workplaces, schools, etc. Shames et al. (2007) broadly classify post-acute interventions into these groups:
Residential community reintegration programs that provide intensive behavioural interventions; Comprehensive day treatment programs that emphasize training in self-awareness, social skills, daily living skills, and coping mechanisms; and Community re-entry programs that focus on vocational and social reintegration.
RTW interventions may be funded and delivered through multiple avenues, but in the U.S., the primary sources of RTW interventions are employer disability management programs and state Vocational Rehabilitation (VR) agencies. Many other countries have equivalent organizations. Brief descriptions of employer disability management and VR are presented in this section.
Employee Disability Management Programs, typically provided directly by employers or through their insurance programs, are a combination of many services (e.g., work injury prevention programs, early intervention, transitional work programs, and case management) to better address the physical and psychological needs of the worker (Ongori, 2012; Shrey, 1996). Job accommodations, support services, and determining the use of adaptive devices are all potential components of disability management (Ongori, 2012; Shrey, 1996). For example, gradual RTW would fall into this category.
As a disability management component, some companies may offer on-site training, which is similar to transitional programs, where the client receives physical, cognitive, and behavioural training while developing job skills in a practice work environment. This allows employers to see how and if they need to make accommodations, and helps the employee develop the skills they need to perform their job (Ben-Yishay et al., 1985; Jacobs, 1997; Klonoff et al., 2007; Prigatano et al., 1994).
State VR Services are provided to individuals with various types of disabilities to increase independence and employment. Services include physical restoration (prosthetic devices), vocational training, counselling, and job development assistance. An increasingly utilized VR service option for clients with TBI is supported employment, where a client obtains paid work that is integrated with people without disabilities with ongoing support from the employer and a vocational specialist (Wehman et al., 2003). These supports may include one-on-one skills training, counselling, and advocacy services by the vocational professional (Wehman et al., 1993). Long-term support services are available to the individual throughout the individual's employment (Wehman et al., 2003). An individual placement supported employment model consists of a vocational specialist or job coach assisting in finding the individual a job, and then this vocational professional coordinates pre-employment and/or on-the-job supports (Wehman et al., 2003).
Although employer disability management programs serve employees who become disabled or are at risk of disability, and VR agencies serve those who are unemployed, there are similarities in the two systems' approaches to RTW. One is the use of job accommodations. Accommodations are negotiated between the employee and the employer. These are alterations made for the employee that do not conflict with the responsibilities of their job, such as flexible scheduling to accommodate episodic disabilities such as epilepsy or psychosocial disorders. Employers may also reassign non-essential duties that the employee cannot perform due to his or her disability. Improved physical accessibility is also a component, such as building ramps for wheelchair access, adding/fixing elevators, relocating an employee's office, widening aisles, and providing wheelchair accessible restrooms and/or grab bars. Also, with the advent of modern technology, high-speed Internet, and webcams, some businesses permit telework from home, allowing an employee to accomplish their daily tasks without having to leave their domicile (West & Anderson, 2005).
Another common strategy is to provide adaptive devices and assistive technology interventions. This can include memory aids, changing from regular to touch-screen based computers, installing adjustable mounts at desks, or utilizing specially-made styli to allow a person with limited fine motor skills to use a computer. With the advent of smartphones and tablet computers, people can use the pre-installed programs to create schedules and set alarms to adhere to them. There are even downloadable applications (apps) created for a number of tasks from detecting anaemia to speech therapy to determining whether people qualify for Medicare benefits (Inge & Targett, 2007).
Finally, both service systems can offer ancillary therapeutic services which, though not directly employment-focused, support the RTW process. These include treatments like physical therapy, occupational therapy, speech/language therapy, cognitive therapy, and peer or professional counselling. Utilizing these programs as a cohesive rehabilitation strategy emphasizes a holistic approach, based on the work of Ben-Yishay (1985).
Purpose of the Review
The most critical outcome from rehabilitating people with TBI is return to work and other productive activities (Fraiser & Clemens, 2000). It is imperative to increase RTW for people with TBI through effective interventions that lead to competitive employment due to its positive impact on the individual, his or her family, and society. Thus, the purpose of this review is to identify and assess the evidence related to interventions for individuals with TBI that are directly focused on RTW. For the purposes of this review, employment interventions will be broadly defined as any strategy, activity, or device that is intended to increase the capacity or the likelihood for an individual with TBI to (a) return to their pre-injury competitive employment position, (b) obtain a new competitive employment position, or (c) retain an existing competitive employment position. Because of the broad array of employment interventions that will be reviewed, each will be assessed separately as to the provider and duration of the intervention.
All intervention studies that will be included will have used competitive employment as the RTW outcome. Our definition of competitive employment will be adapted from that of the U.S. Department of Education's Rehabilitation Services Administration (RSA). The RSA definition includes a criterion that the person be paid the prevailing minimum wage or greater. We will not include this criterion because we will be conducting an international search, and some nations may not have an established minimum wage. Our definition of competitive employment will therefore be based on three criteria:
Work is performed by the individual with TBI in an integrated setting, (i.e., not a disability organization, co-workers predominantly non-disabled); The employee is paid by the business or organization where the work is performed, not by a disability organization (such as in enclaves); and The employee is paid commensurate with non-disabled co-workers having similar qualifications, performing similar duties.
Although individuals with TBI and other disabilities may access other employment options (i.e., sheltered employment, volunteer positions), competitive employment was singled out as the optimal RTW outcome. The value of non-competitive employment services such as sheltered workshops and day vocational programs has been a hotly debated issue for many years (Cimera, Wehman, West, & Burgess, 2012; Migliore, Mank, Grossi, & Rogan, 2007). Without adding to this debate, it is self-evident that getting and keeping a competitive position and a non-competitive position are different processes and differ greatly in their demands on service providers, families, and persons with disabilities themselves. As one example, successful entry into a sheltered work program would typically hinge upon an available service slot and a funding source, whereas in the labor market persons with disabilities must compete for positions against non-disabled applicants and would need to show that they add value to the business. The disparity between competitive and non-competitive environments is supported by a recent study by Cimera, Wehman, West, and Burgess (2012), which found that sheltered work experience had no effect on competitive employment outcomes for state VR clients with autism. Similarly, work performance and value of individuals in volunteer positions are not evaluated as critically as that of paid employees, and therefore it is more likely that a low-performing volunteer will be retained than a low-performing employee. To summarize, competitive employment is the outcome of interest because non-competitive employment options do not reflect the true demands of entering and remaining in the workforce and are therefore of less interest in assessing RTW interventions.
How the Intervention Might Work
Employment barriers and interventions may be broadly classified as either supply-side or demand-side (Chan, Strauser, Maher, Lee, Jones, & Johnson, 2010). Supply-side barriers are those related to the individual with disabilities, such as deficits in skills, interfering behaviors, functional limitations related to the disability, and accommodation needs. Thus, supply-side interventions address those issues via skill or behavioral training, accommodations, and provision of other needed support to the individual with disabilities and his or her family members. Supply-side interventions may be delivered by therapists, educators, employment specialists, and VR Counselors, with durations from days to years depending on the nature of the intervention. For example, an individual in a supported employment program may theoretically receive ongoing support throughout his or her working history.
Alternatively, demand-side barriers are those that are related to the employer and the place of business. Examples of these barriers might include limited experience with employees with disabilities, high turnover or absentee rates, high production demands, and high costs related to injured workers. Thus, demand-side interventions may include disability management practices such as gradual return to full duty following injury, employer training and technical assistance related to disability and accommodation, co-worker and supervisor support and assistance, and employer-provided internships, skills training, or support programs. Demand-side interventions are typically delivered by either Vocational Rehabilitation agencies or employers/businesses themselves.
The Logic Model shows the connections between supply- and demand-side interventions, recipients of those interventions, and anticipated outcomes. For this review, we will include both types of interventions for individuals with TBI.
Why it is Important to do the Review
Previous systematic reviews have examined the effectiveness of specific interventions on persons with TBI regarding their community integration (Kim & Colantonio, 2010); cognitive rehabilitation (Cicerone et al., 2011; Rohling, Faust, Beverly, & Demakis, 2009); quality of life; functional independence; physical, psychological, and social functioning; and community participation (Cattelani, Zettin, & Zoccolotti, 2010; Evans & Brewis, 2008). Van Velzen, Van Bennekom, Edelarr, Sluiter, and Frings-Dresen (2009) examined the number of people with acquired brain injury who returned to work for a 16-year period but did not look at vocational rehabilitation interventions. Similarly, Nightingale, Soo, and Tate (2007) conducted a systematic review that examined the prognostic factors related to people with TBI RTW. Two of their outcome variables were productivity and competitive employment; however, Nightingale et al. (2007) did not examine interventions. There have also been systematic reviews that have examined the effectiveness of interventions on vocational outcomes of those with acquired brain injury (ABI) (Turner-Stokes, Nair, Sedki, Disler, & Wade, 2005). A recent Campbell Collaboration title registration by Braathen et al. (2011) proposes to examine the effectiveness of comprehensive occupational rehabilitation programs for RTW among people on long-term sickness absence. Their review will include people with mental disorders and/or musculoskeletal disorders.

Logic Model: Employment interventions for return to work in working aged adults following traumatic brain injury (TBI)
Further, Fadyl and McPherson (2009) reviewed and evaluated the evidence for effectiveness, strengths, and weaknesses of vocational interventions for persons with TBI. Unlike the longer time period (1973 to 2011) and broad information retrieval (more databases and grey papers) of this proposed review, Fadyl and McPherson's (2009) electronic database-limited search focused on articles available in English from January 1990 to July 2007 in Ovid Journals database, which includes MEDLINE, PsychInfo, CINAHL, AMED, Health and Psychosocial Instruments, Evidence -Based Medicine databases, and Web of Science. Geurtsen, Van Heugten, Martina, and Geurts (2010) conducted a systematic review of effectiveness of comprehensive rehabilitation program interventions on a number of outcome variables including employment. Only PubMed, PsychINFO, and PsychLit databases and articles published from 1990 to 2008 were used. Similarly, a recent systematic review protocol that addresses the effectiveness of multidisciplinary post-acute rehabilitation for adults with moderate to severe TBI (Agency for Healthcare Research Quality, 2011) limits its electronic database search to MEDLINE, the Cochrane Central Register of Controlled Trials, PEDro, and PsychINFO.
This proposed study will search a wider range of databases, search and include grey literature, search a broader length of time (1973 to 2011), and focus on competitive employment outcomes among persons with TBI regardless of injury severity. By including all competitive employment, supply- and demand-side, evidence-based research interventions for persons with TBI, this systematic review will provide clarity on the effectiveness of these employment intervention programs that will guide practitioners and vocational rehabilitation policy makers. Further, this analysis will provide information concerning the direction for future research.
OBJECTIVES
This systematic review will examine the effectiveness of interventions on RTW outcomes of adults with TBI. The goal of this project is to provide information that will benefit people with TBI, employers, and vocational professionals by informing these audiences of the effectiveness of employment interventions. The information we obtain from this analysis may indicate research gaps that provide direction for future research.
Methodology
The methodology found in this protocol is guided by the Campbell Collaboration Guidelines on Systematic Review Methods http://www.campbellcollaboration.org/artman2/uploads/1/C2_Protocols_guidelines_v1.pdf (retrieved September 27, 2012).
Criteria for Including Studies in the Review
Study Inclusion Criteria
Types of study designs. Studies that use randomized controlled trials (RCTs) and quasi-experimental designs will be included. Comparison group conditions will involve treatment as usual, other appropriate interventions, or no intervention.
Types of participants. Studies will be included that have participants who are of working age (18 to 65 years of age), have a closed TBI (injury which did not penetrate the brain) diagnosis regardless of severity, are engaged in full- or part-time competitive employment at the time of injury, injured either on the job or off the job, are unemployed, or are on medically related leave through the employer at the time of intervention receipt. Studies can include all times since injury. Participants can be injured either on or off the job, and subgroup analysis will be conducted for those injured on the job as opposed to those that were not. Studies that have samples with mixed disability groups will be included if and only if the researchers provide data for the TBI participants separately from the non-TBI participants. In situations in which non-competitively employed students and participants who are engaged in competitive employment are mixed in a study, we will use data if and only if data for the TBI participants who are competitively employed are presented separately from the non-competitively employed students in the study results.
Types of interventions. Studies will be included that use interventions that assist working age adults with TBI in returning to competitive employment, including self-employment (with competitive employment as the outcome). The intensity of the intervention can vary widely. Interventions implemented in any country will be eligible for inclusion so long as the study is written in English or Spanish. On the Coding Form (Appendix 3), we will code for type of intervention (e.g., transitional, supported, holistic) used and whether the intervention was demand- or supply-side. We will also note on the coding form whether a business, state vocational agency, psychologist, etc. carried out the intervention.
Outcome measures. The RTW outcome is continuous competitive employment, also referred to as open or independent employment, in which individuals with TBI have been full-time or part-time employed for 45 days or more. The premise of using competitive employment as the one outcome variable is that competitive employment represents optimal real world functioning. Our definition of competitive employment will be based on three criteria:
Employed individuals with TBI perform employment related tasks in an integrated setting (i.e., not a disability organization) and must interact with co-workers who are predominantly non-disabled. Employed individuals with TBI are paid by the business or organization where the work is performed, not by a disability organization (such as in enclaves). In this way, competitively employed individuals with TBI have the same risks of losing their position as non-disabled individuals. Employed individuals with TBI are also paid commensurate with non-disabled co-workers having similar qualifications and performing similar duties. When an individual with TBI is competitively employed, the individual vied for a position against individuals without disabilities, which is more difficult than sheltered employment or volunteer work, and in most cases, more difficult than entering an educational program. In contrast, good consistent performance is not required for volunteer positions and sheltered workshops. Opportunities to re-do their performance through retaking exams or courses are not typically available in competitive work environments.
The primary outcome variable will be a dichotomous variable—employment status—in which we will use the proportion of participants in each study who are employed and those unemployed. There may be other secondary continuous outcomes such as length of time employed and mean hours worked.
Study Exclusion Criteria
The following will be excluded from the review:
Studies that do not examine the effectiveness of interventions for individuals with TBI disabilities that support RTW and/or retention of competitive employment. Studies where participants' are only students. Single-subject designs and other studies that are not RCTs or quasi-experimental designs. Studies that do not use interventions. Studies that do not report competitive employment as an outcome.
Search Strategy
Database Search
All relevant studies that meet the inclusion criteria will be identified and retrieved. A number of strategies will be used to obtain both published and unpublished studies that (a) identify interventions that assist persons with TBI to return to competitive work and (b) assess the strength of the evidence of the empirical evidence for them. We will restrict the search to articles and grey-literature papers to those written in either English or Spanish that were published between 1973 and 2011. To include reports in all languages would be untenable and fiscally burdensome to include in this report.
Computerized database searches relevant to rehabilitation health care, social sciences, and business will be conducted. A comprehensive database search will include the following databases:
Academic Search Complete Article First Australia Education Index British Educational Index Business Source Complete CINAHL Plus with full text CIRRIE (Center for International Rehabilitation Research Information and Exchange Database) Cochrane Central Registry of Controlled Trials ClinicalTrials.gov Controlled-Trials.com COS Conference Papers Index COS Scholar Universe Directory of Published Proceedings Dissertations and Theses Full Text (Proquest) ebrary: Academic Complete Collection EBSCOhost Research Database Education Research Complete ERIC FirstSearch Databases FRANCIS HealthSource: Nursing/Academic Edition Google Scholar Lexis-Nexus Academic MEDLINE/PubMed NARIC REHABDATA PapersFirst ProQuest PsycExtra PsychINFO Psychology and Behavioral Sciences Collection Science Direct Sociological Abstracts Web of Science World Cat
The search will be conducted for studies published between 1973 and 2011. Based on Humphrey and Oddy's 1980 review of RTW in TBI populations from 1954 to 1978, it is very unlikely that we will retrieve any studies of vocational intervention in the form of experimental, quasi-experimental, and single-subject experimental designs published before 1980. Most of the studies addressed prognosis of RTW. However, in order to ensure that we are able to retrieve all appropriate studies for our systematic review, we have used as our starting point the year 1973, when the Rehabilitation Act was passed, which prohibited discrimination due to disability by federal agencies and entities receiving federal financial assistance. All searches will be recorded per the Literature Search Protocol (Appendix 1) in Excel file, Search Record (Appendix 2).
Hand Searches
We will examine reference sections of included studies, as well as other relevant studies and documents, to determine whether there are relevant cited reports that were found through the electronic database searches.
Grey Literature
In addition to the computerized database searches, grey literature will be located and retrieved by searching websites maintained by state, federal, and international organizations that have conducted research related to the topic. These include government agencies, research organizations, foundations, clearinghouses, and archives. Conference Proceedings Citation Index, Directory of Published Proceedings, and PapersFirst will also be searched for leads to potential studies. A list of websites is listed in Appendix 1 and includes the following examples:
Bibliographies
Reference lists of included studies and related systematic reviews on the topic will be examined to locate unpublished studies. Grey literature will undergo the same inclusion criteria as other studies identified through the electronic database searches.
Search Terms
Where available, database thesauri will be consulted to ensure that appropriate synonyms have been included in the intervention, population, and outcome search categories. Search strings (keywords and Boolean operators) will be created so that the search strategy is transparent and replicable. The search strategy will be tailored for each database, as each database will require a slightly different strategy, and not all terms will be used in each search. We will begin each search in databases using search strings found in Appendix 1 (Literature Search Protocol) and then expand search strings based on the database. We will search TBI by TBI, head injury, brain injury, and acquired brain injury. Since other injuries besides TBI are found in the brain, head, and acquired brain injury literature, note that TBI participants will be included if and only if researchers provide data for the TBI participants separately from other types of brain/head or acquired brain injuries.
Description of Methods Used in Primary Research
Studies in the systematic review will consist of randomized controlled trials (RCTs) and quasi-experimental designs. For those few studies that are RCTs, we will use post-test proportion of participants employed from the intervention and comparison group. We anticipate that most studies will be quasi-experimental and also not have comparison groups. We plan to examine the pre-post effect size of all interventions that report the same type of outcomes. Thus, all studies using the primary outcome variable—proportion of participants employed—and having pre- and post-test scores will be analysed together. Later in this section, we discuss how we will address those studies that have multiple outcome time points (post-intervention and follow-up).
Comparison group conditions will involve treatment as usual, other appropriate interventions, or no intervention. We will analyse studies together that have the same type of comparison groups. Studies in which there are intervention groups and control groups that do not receive an intervention or service will be analysed together. All comparison groups in multi-comparison studies will be reported on The Coding Form for Primary Research Characteristics and in the Table of Characteristics of Primary Research. The Coding Form for Primary Research Characteristics (Appendix 3) will capture primary research characteristics of each study.
We have included an example of a coded study that will be included in the systematic review in Appendix 4. The report is by Ellerd and Moore (1992) on a supported employment intervention, which used one group post-test only with follow-up design and data collection occurring at 12 and 30 months. Dichotomous and continuous variables were used as outcome variables in this study. The dichotomous variables consisted of number of participants employed at 12 and 30 months. The continuous variables consisted of length of time employed, mean hours worked, and mean wages at 12 and 30 months. The primary outcome for this systematic review is the proportion of participants who are employed.
Criteria for Determination of Independent Findings
Multiple studies that use the same sample or data will be treated as a single study, and information from these studies will be combined so that any data that is relevant will be used in this project. Original authors will be contacted in an attempt to clarify any conflicting information amongst the studies. Only one score will represent these studies.
Multiple Outcome Studies
Any secondary outcome variables that are analysed in a study will be represented by one score. When there are multiple time points for secondary outcome variables, the mean for the variable will be used. For instance, in the Ellerd and Moore (1992) study, the mean of the length of time employed of the 12 and 30 months length of time will be used.
Multiple Outcome Time Points
When there are multiple outcome time points in the study, we will take the mean proportion across outcome scores (post-intervention and follow-ups) to determine the overall effect size for that study. For instance, in the Ellerd and Moore (1992) study, the mean of the number of 12 month employed participants and 30 month employed participants will be used.
Multiple Comparison Groups Studies
It is unlikely that there will be multiple comparison groups. However there is a case in which the overall study was reported through multiple articles, and there two distinct intervention programs that were used with separate intervention samples (Braverman et al., 1999; Salazar et al., 2000; Warden, Salazar, Marin, Schwab, Coyle, & Walter, 2000). We will analyse data separately for each intervention.
Details of Study Coding Categories
Selection of Studies
Determination of inclusion and exclusion of studies will be conducted in two stages. In Stage One, the title and the abstract of all the references of the studies that have been identified will be obtained and screened by two reviewers to determine whether they meet the inclusion and exclusion criteria of this review. If both reviewers agree that the study meets the inclusion criteria, the full-text of the study will be obtained. If the reviewers are unsure, the citation will proceed to the full-text eligibility screening stage for a final inclusion decision.
In Stage Two, the full-text eligibility screening stage, two reviewers will independently code the full-texts of each study that proceeded to the full-text stage, using the coding form developed for this review (see below and Appendix 3 for coding form), to determine whether the study will be included or excluded from the review.
The coding form (Appendix 3) will track publication source, research design, subject characteristics, sample source, employment setting, intervention characteristics (e.g., number of participants employed at post-test, type and duration of services provided), and outcome measurements (e.g., proportion employed at post-test and duration of continued employment). Eligible studies will be coded for methodological qualities, such as allocation concealment, blinding, sequence generation, incomplete outcome data, and selective outcome reporting. While blinding is challenging and not always feasible in rehabilitation and behavioral studies (Friedberg, Lipsitz, & Natarajan, 2010), it will be coded if blinding occurred.
If either reviewer was involved in any of the studies, a third reviewer will code that study to minimize bias. Inter-rater reliability will be established between the two reviewers prior to beginning coding activities. Reasons for the decision will be noted for each study. Notes for each study of both coders will be compared. If both coders agree on the eligibility decision for a study, the study will be included in the review. The reviewers will independently record all excluded studies along with the reasons for exclusion. If there is a disagreement in coding, reviewers will strive for consensus. If consensus is not reached, then a third reviewer will be asked to decide the coding conflict.
Quality Assessment
Two project reviewers (one with methodological expertise and one with content expertise) will code studies for methodological quality. These coders will pilot test the coding protocol to establish inter-rater agreement before coding studies for this project.
We will use the Cochrane's risk of bias tool and assess five factors this device addresses: sequence generation; allocation concealment; blinding of participants, personnel, and data collectors; incomplete data; and selective outcome reporting (Higgins & Altman, 2008). We will code these factors as low risk, high risk, and unclear/unknown. For these factors, we will provide a Risk of Bias Table and Risk of Bias Summary. Other data will be extracted design, review process (i.e., peer reviewed, not peer reviewed, unknown), type of publication (e.g., journal article, book chapter, dissertation, paper presentation), and presence of control group (Higgins & Altman, 2008; Wortman, 1994). Percentages and frequencies will be provided for all factors. Furthermore, all factors will be analyzed as sub-group analyses to assess their potential explanatory impact on the effect of the intervention.
Statistical Procedures and Conventions
We anticipate that a small number of studies will be included in this systematic review. However, we expect that the number of studies will be adequate to conduct a meta-analysis. Comprehensive Meta-Analysis 2 (CMA2; Biostat, 2005) will be used to conduct the meta-analysis. We will conduct an overall meta-analysis combining the experimental and quasi-experimental designs first. This will be followed by subgroup analyses including examining experimental studies separately from quasi-experimental. Next, we will determine how type of comparison group influences the effect size. We will use random effects models to conduct the meta-analysis, with inverse variance weighting of studies. Forest plots will be used to display effect size, 95% confidence intervals, and weighting.
Characteristics of Primary Literature
We will develop a descriptive table—Table of Characteristics of Primary Research—of all the included studies in the analysis. This table will include participants, sample size, description of the intervention, outcome variables, and observed effect size.
Effect Size for Binary Variables
The primary outcome variable is binary competitive employment outcomes (e.g., presence or absence of gaining competitive employment). Most employment intervention studies report whether or not participants in the intervention and control group obtained employment. We will report odds ratios for binary outcomes including the primary outcome. In the analyses, we will compute log odds ratio and standard error of the log odds ratio for binary outcome variables and then convert back to odds ratios. We do not anticipate having to convert odds ratios to Cohen's d.
Effect Size for Continuous Variables
There are two expected secondary outcome variables that are continuous: mean hours worked and mean length of time employed. The effect sizes will be computed for this meta-analysis using the commonly understood standardized mean difference (Cohen's d). To correct for small sample bias, Cohen's d will be converted into Hedges g.
Missing Data
Situations in which outcome data is incomplete, we will attempt to contact the designated contact author and/or the first author to request the missing data.
Publication Bias
We will be conducting a grey literature search. However, to determine whether there are missing studies, a funnel plot will be used to determine any evidence of publication bias. If there is bias, the trim and fill procedure will be used to adjust for publication bias.
Heterogeneity
Heterogeneity of included studies will be assessed by using (1) a chi-squared test to determine whether Q is significant and (2) the I2 to determine the proportion of variance that is due to heterogeneity (Borenstein, Hedges, Higgins, & Rothstein, 2009). A significant Q (p < 0.05) and a moderate I2 of 0.50 will be the criteria for determining heterogeneity (Borenstein et al., 2009).
Subgroup Analysis
We will conduct subgroup analyses when there is adequate number of studies. Actual moderators that will be used if there is sufficient number of studies will be determined by consensus upon review. Potential moderators include (1) level of education pre-injury, (2) severity of TBI, (3) type of intervention, (4) provider (demand- or supply-side), (5) duration of intervention using 6 months increments (6 mos. or less, 6.1 mos. to 12 mos., etc.), (6) subject population (civilian or military), (7) where injury occurred (fall, motor vehicle/traffic, colliding with an object, assault, combat/terror), and (8) type of comparison group (no intervention, treatment as usual, other intervention).
Sensitivity Analysis
Sensitivity analysis will be conducted to determine the robustness of findings of the pooled effect sizes examining the impact of: (1) each study on the overall summary effect, (2) separately analysing randomized control trials from quasi-experimental studies, and (3) chronological order of studies.
Treatment of Qualitative Research
No qualitative research will be included in this meta-analysis. However, we will review relevant qualitative research that concerns the results of the meta-analysis. The qualitative research will be used if it enhances the discussion of the quantitative meta-analysis. This review will be entirely separate from the meta-analysis. This discussion will be presented in a separate section within the discussion.
REVIEW TEAM
Title: Research Methodologist, Associate Professor
Affiliation: Virginia Commonwealth University, Center for Rehabilitation Sciences and Engineering, Department of Physical Medicine and Rehabilitation
Address: P.O. Box 843038
City, State, Province or County: Richmond, Virginia
Postal Code: 23298-3038
Country: United States
Phone: 804.827.6978
Mobile: 804.501.8899
Email:
Title: Director of Research
Affiliation: Virginia Commonwealth University, Rehabilitation Research and Training Center
Address: P.O. Box 842011
City, State, Province or County: Richmond, Virginia
Postal Code: 23284-2011
Country: United States
Phone: 804.828.5957
Mobile:
Email:
ROLES AND RESPONSIBLIITIES
Content: Michael West has expertise in the field of vocational rehabilitation, including RTW of individuals with TBI. Systematic review methods: Carolyn Graham has conducted, presented, and published meta-analyses. We will also seek guidance from Dr. John Westbrook from SEDL as necessary. Statistical analysis: Carolyn Graham is a research methodologist involved in designing studies, conducting power analyses, and collection and analyzing data for rehabilitation/disability studies across a number of disciplines such as medicine, education, rehabilitation counseling, psychology, and bio-engineering. Her skills in conducting quantitative research include conducting meta-analyses. She has over 16 years of experience in teaching research methods, statistics, and qualitative research. Information retrieval: We will seek guidance as necessary from a SEDL librarian in locating publications catalogued in US databases. For international literature retrieval, the team will seek assistance from the ECG.
SOURCES OF SUPPORT
This study was funded by Center on Knowledge Translation for Employment Research through grant #H133A100026 to SEDL from the National Institute on Disability and Rehabilitation Research (NIDRR) in the U.S. Department of Education's Office of Special Education and Rehabilitative Services.
REQUEST SUPPORT
The reviewers may seek guidance and assistance in systematic searches of international databases. Only those studies written in English will be included.
DECLARATIONS OF INTEREST
There are no conflicts of interest.
PRELIMINARY TIMEFRAME
Once the protocol is approved, authors expect to submit the first completed draft of the review within six months.
PLANS FOR UPDATING THE REVIEW
Authors will assess updating the review every three years.
AUTHORS' RESPONSIBILITIES
By completing this form, you accept responsibility for preparing, maintaining and updating the review in accordance with Campbell Collaboration policy. The Campbell Collaboration will provide as much support as possible to assist with the preparation of the review.
A draft review must be submitted to the relevant Coordinating Group within two years of protocol publication. If drafts are not submitted before the agreed deadlines, or if we are unable to contact you for an extended period, the relevant Coordinating Group has the right to de-register the title or transfer the title to alternative authors. The Coordinating Group also has the right to de-register or transfer the title if it does not meet the standards of the Coordinating Group and/or the Campbell Collaboration.
You accept responsibility for maintaining the review in light of new evidence, comments and criticisms, and other developments, and updating the review at least once every three years, or, if requested, transferring responsibility for maintaining the review to others as agreed with the Coordinating Group.
PUBLICATION IN THE CAMPBELL LIBRARY
The support of the Campbell Collaboration and the relevant Coordinating Group in preparing your review is conditional upon your agreement to publish the protocol, finished review and subsequent updates in the Campbell Library. Concurrent publication in other journals is encouraged. However, a Campbell systematic review should be published either before, or at the same time as, its publication in other journals. Authors should not publish Campbell reviews in journals before they are ready for publication in the Campbell Library. Authors should remember to include the statement: “This is a version of a Campbell review, which is available in The Campbell Library” when publishing in journals or other venues.
I understand the commitment required to undertake a Campbell review, and agree to publish in the Campbell Library. Signed on behalf of the authors:
Form completed by: Carolyn W. Graham, PhD
Date: 31 October 2013
