Abstract
Adults with Autism Spectrum Disorders (ASD) may face challenges in employment settings due to difficulties in social functioning. Bespoke employment interventions aim to make it easier for adults with ASD to secure and maintain jobs. This Campbell systematic review finds that there is not yet enough high quality research to draw a conclusion on their effectiveness.
This review focused on employment interventions for adults 18 years or older with a diagnosis of ASD, and who were no longer enrolled in a school-to-work or secondary-level education programs. The review considered interventions that centered on competitive, supported or integrated employment but did not include those where the treatment groups were not in an integrated or mainstream format of employment.
The review focused on one outcome, attainment of a job placement, based on specific information about the duration and/or retention of that placement. One of the two included studies looked at a program where support workers gave guidance on job searching, work preparation and employer communication. The second study looked at supported employment involving jobs located in the local community and guidance from job coaches.
Executive summary/Abstract
BACKGROUND
The incidence of Autism Spectrum Disorders (ASD) has been increasing over the last two decades. Currently, 1 in 110 children are identified with ASD in the United States (CDC, 2009). ASD refers to a range of neurological disorders that involve some degree of difficulty with communication and interpersonal relationships. The range of the spectrum for autism disorders is wide with those at the higher functioning end often able to lead relatively independent lives and complete academic programs even while demonstrating social awkwardness. Those at the lower functioning end of the autism spectrum often demonstrate physical limitations, may lack speech, and have the inability to relate socially with others.
As persons with ASD age, options such as employment become increasingly important as a consideration for long-term personal planning and quality of life. While many challenges exist for persons with ASD in obtaining and maintaining employment, some research is showing that, with effective behavioral and social interventions, employment can occur (Schaller & Yang, 2005). In fact, about 37% of individuals with ASD report having been employed for 12 months or more, four years after exiting high school (Newman, Wagner, Cameto, & Knokey, 2009). However, several studies show that individuals with ASD are more likely to lose their employment for behavioral and social interaction problems rather than their inability to perform assigned work tasks (Dew & Alan, 2007; Hurlbutt & Chalmers, 2004; Unger, 1999).
Research has been conducted in the area of autism and employment, but such research has not been reported in a comprehensive evidence review format before. Thus, a systematic review on the effectiveness of adult employment assistance interventions for individuals with ASD would not only demonstrate the extent and magnitude of the interventions' effects, but also provide ideas for further research that can inform implementation and refinement of related employment-focused programs.
OBJECTIVE
The objective of this review is to determine the effectiveness of adult employment interventions in securing and maintaining employment for adults with ASD.
SEARCH STRATEGY
The comprehensive search strategy used to identify relevant studies included the review of 28 relevant electronic databases. Search terminology for each of the electronic databases was developed from available database thesauri. Appropriate synonyms were used to maximize the database search output. Several international databases were included among the 28 databases searched.
In addition, the authors identified and reviewed grey literature through analysis of reference lists of relevant studies. Unpublished dissertations and theses were also identified through database searches. The programs of conferences held by associations and organizations relevant to ASD and employment were also searched.
SELECTION CRITERIA
A two-stage process was used to determine inclusion or exclusion of studies: (1) two reviewers independently reviewed the title and abstract of each identified study, and (2) for studies advanced from the first stage, two independent reviewers reviewed the full text of each study for final determination of inclusion or exclusion. Inclusion criteria allowed the following research designs: randomized controlled, quasi-experimental, and single subject experimental designs.
DATA COLLECTION AND ANALYSIS
The electronic searches of databases yielded 8,528 citations for the first stage of review. Of these, a total of 77 citations were selected for second stage full-text review of each study. Upon review of the full-text for each of the 77 studies, two studies were retained, having met the inclusion criteria.
Of the two included studies, both were quasi-experimental research designs. The studies generally described the effects of a supported employment intervention for adults with ASD on either employment outcomes or aspects of cognitive functioning. The nature of the data provided did not lend itself to a traditional meta-analysis. Given the number of studies, study designs, and the diversity of outcomes across the two studies, it was not possible to aggregate results across studies.
RESULTS
This review was not able to identify definitive interventions that predictably and positively supported the development of employment outcomes for individuals with ASD.
There were two included studies. One, Mawhood and Howlin (1999), described outcomes directly related to employment by comparing an experimental group (n = 30) who received guidance from a support worker in the form of job finding, work preparation, and communication with the employers with a control group (n = 20) that did not receive any support. At the end of the two-year study period, the authors found that the experimental group demonstrated significantly higher rates of having found paid employment (d = 1.067, 95% CI = 0.123 to 2.010), significantly longer periods of working time (d = 0.684, 95% CI = 0.111 to 1.257), and significantly higher wages (d = 1.177, 95% CI = 0.169 to 0.819). The experimental group also worked more hours per work week (d = 0.328, 95% CI = -0.628 to 1.284), but this effect was not significant.
The second included study, Garcia-Villamisar, Ross, and Wehman (2000), described differential effects of supported employment and sheltered employment for persons with autism. The experimental group received supported employment, i.e., jobs working between 15 and 30 hours per week in their communities and received job coach support services. The control group received sheltered employment, i.e., jobs that were not in their communities and with no job coaching services. Using measures of the Childhood Autism Rating Scale (CARS), the supported employment group was not significantly different from the sheltered employment group after treatment (d = -0.229, 95% CI = -0.764 to 0.306). The study did not report any differences in employment acquisition or maintenance between the two groups.
In an analysis of risk bias of the included studies, it was found that there was a high level of substantial potential for bias across three of the five sources of bias analyzed including unit of assignment, unit of analysis, attrition, fidelity of implementation, and blinding. The methodological quality of both included studies was low. Therefore, these review results should not be interpreted as indicating definitive results related to the development of employment outcomes for persons with ASD.
AUTHORS' CONCLUSIONS
Qualitative and other relevant research studies connected to the employment of persons with ASD were also reviewed and suggest that the following may be elements of successful employment placement for persons with ASD: (1) identification of the most appropriate work settings and placements, (2) provision of effective supports on the job, (3) need for long-term support services for the employer and the consumer, (4) costs for support, and (5) positive effects of employment on persons with ASD. While qualitative studies point to a number of promising issues for future research, they do not provide a definitive statement about what works.
In addition, the authors noted that costs for community-based employment interventions such as those included in this review are more expensive than other employment alternatives such as sheltered non-integrated workshops. However, Howlin, Alcock & Burkin (2005) and others (e.g., Cimera & Cowan, 2009) show that supported employment service interventions, such as those identified in our included studies, are becoming less expensive to deliver. It seems important to note that community-based integrated employment interventions, while expensive, do expand options for mainstream social integration, competitive wages, and community involvement.
1 Background
The first paper identifying a condition called autism was published in 1943 (Kanner, 1943). Today, the condition is referred to as autism spectrum disorders (ASD) and refers to a range of mild to severe forms of neurological disorders marked by impairment in social functioning, communication, and repetitive and unusual patterns of behavior.
According to the Centers for Disease Control and Prevention (CDC), the rate of autism has increased over the past two decades to 1 in 110 children currently being identified with autism spectrum disorder (CDC, 2009). More recently, research indicated rates as high as 2.64% of individuals with ASD in the population (Kim et al., 2011). Functional limitations caused by ASD continue into adulthood and often create barriers to independent living and stable long-term employment (Autism Society of America, 2008).
ASD refers to a range of neurological disorders that involve some degree of difficulty with communication and interpersonal relationships, as well as obsessive and repetitive behaviors. ASD occurs in all racial, ethnic, and socioeconomic groups. There is a wide range of effects demonstrated across the spectrum. Those at the lower functioning end of the spectrum often demonstrate physical limitations and may not be able to speak or relate socially to others. Those at the higher functioning end of the spectrum are often able to lead relatively independent lives, graduate from academic institutions, but may also be awkward in their social interactions and have difficulty developing friendships. Some less frequent disorders such as Rett Syndrome affect mostly girls, while Childhood Disintegrative Disorder affects mostly boys; in both cases, there is a period of normal development before the onset of severe autistic and other health-related symptoms. Men manifest ASD at a rate four to five times higher than women. The significant gap represented from the high to the low end of the spectrum dramatically affects employment-related skills, abilities, and behaviors (Centers for Disease Control and Prevention, 2011).
Given the increase of ASD prevalence and the number of students with ASD exiting public school settings, attention is increasingly focused on potential employment outcomes for individuals with ASD. Individuals without severe disabilities are eight times more likely to be employed than individuals with very severe disabilities (National Organization on Disability, 2000). Individuals with ASD are among those least likely to be employed (Cameto, Marder, Wagner, & Cardoso, 2003; Dew & Alan, 2007); in fact, only 15% of individuals diagnosed with ASD in the United States gained employment (Cameto, Marder, Wagner, & Cardoso, 2005).
Although economic conditions and employer attitudes affect employment opportunities, employment outcomes for individuals with ASD can be improved by appropriately addressing specific behaviors common among individuals with ASD (Schaller & Yang, 2005). Several studies have indicated that individuals with ASD are more likely to lose their employment for behavioral and social interaction reasons rather than their inability to perform work tasks (Dew & Alan, 2007; Hurlbutt & Chalmers, 2004; Unger, 1999). Behavior management challenges affecting employment must be understood and addressed consistently by employment support service providers in order to effectively facilitate obtaining and maintaining employment by individuals with ASD. However, relatively few employment support service providers including state vocational rehabilitation counselors have an in-depth understanding of services that are associated with developing successful employment outcomes for individuals with ASD (Dew & Alan, 2007).
Each state in the United States has at least one state agency that is charged with the provision of vocational rehabilitation services to facilitate employment outcomes for eligible individuals with disabilities. Data accumulated through the national network of state vocational rehabilitation agencies suggest that few individuals with autism were requesting services and of those that do most were not successfully employed as an outcome of the service. However, in its latest report, the National Vocational Rehabilitation Service System indicated that in fiscal year 2007 it provided services that successfully placed 1,774 individuals with ASD into employment situations that continued for 90 or more days (Rehabilitation Services Administration, 2009).
1.1 CONTRIBUTION OF THE REVIEW
Research has been conducted in the area of autism and employment, but such research has not been reported in a comprehensive evidence review format before. Thus, a systematic review on the effectiveness of adult employment assistance interventions for individuals with ASD would not only demonstrate the extent and magnitude of the interventions' effects, but also provide ideas for further research that can inform implementation and refinement of related employment-focused programs.
More needs to be known about strategies that are successful in developing and maintaining employment outcomes by individuals with ASD. As the population of individuals with ASD grows, enhanced awareness of effective approaches is likely to increase the availability of more effective employment support services, such as job development and tailoring, job coaching, vocational rehabilitation, and supported employment. A glossary to assist readers in understanding the differences in terminology related to different types of employment outcomes and job support interventions is provided in Appendix A: Potential Job Supports and Interventions Glossary. The present review of studies on adult employment assistance interventions for individuals with ASD will produce a synthesis of the intervention impact as well as highlight the gaps in experimental research in this area.
2 Objectives
To determine the effectiveness of adult employment assistance in securing and maintaining employment for adults (18 years and older) with autism spectrum disorders (ASD).
3 Methods
3.1 CRITERIA FOR INCLUSION AND EXCLUSION OF STUDIES IN THE REVIEW
A two-stage process was used to determine inclusion or exclusion of studies: (1) title and abstract stage and (2) full text stage.
3.1.1 Title and Abstract Stage
Studies were assessed and selected for advancement to the next stage of inclusion if evaluation meets at least one of the two following criteria:
3.1.1.1 Participants
The participant sample of study included only adults 18 years or older with a diagnosis of ASD, who were no longer enrolled in a school-to-work program or secondary-level education program. The review was designed to identify effective interventions that could be implemented by adult service program agencies; therefore, secondary school and other employment programs conducted in school settings with students were excluded. The authors are conducting another systematic review focusing on the effectiveness of employment assistance interventions for transition-age youth with ASD currently enrolled in an educational setting context.
3.1.1.2 Intervention
The focus of the study intervention centered on the topic of employment. The types of employment included were competitive, supported, or integrated employment. Studies in which the experimental groups assigned to sheltered work or non-integrated work interventions were excluded from the review due to not providing the integrated or mainstream format of employment. Studies that reported effect sizes were included in the review.
If these criteria were not clear from the title or abstract, the study was advanced for retrieval of the full text to determine eligibility.
3.1.2 Full-Text Stage
Full-texts of studies from all citations/abstracts advanced from Stage 1 were retrieved for a final determination of inclusion in the review and analysis. All of the following criteria were required for each study in order to be included in the review and analysis.
3.1.2.1 Research Design
Studies used an experimental or randomized controlled trial design (RCT), quasi-experimental design (QED), or single subject experimental design (SSED) to report the effects of the intervention.
3.1.2.2 Participants
The recipients of the intervention were individuals with an ASD, 18 years of age or older and were voluntarily seeking assistance in obtaining employment. Study participants with ASD eligible for inclusion were individuals with Asperger Syndrome, Autism, Rhett Syndrome, Childhood Disintegrative Disorder, or Pervasive Developmental Disorder-Not Otherwise Specified, as defined in the DSM-IV-TR and diagnosed by an appropriate professional.
Participants not employed at the time of the study intervention were the focus of this review. Reviewers did not exclude studies in which the participant pool included both participants who had an employment history and those who did not. Individuals who were employed prior to an intervention study were not excluded in this review.
Study participants with ASD and other secondary disabilities were included; however, study participants with sole disabilities such as mental retardation, schizophrenia, attention deficits or other non-autism related conditions were not included.
3.1.2.3 Intervention
The interventions were required to provide adult employment assistance intended to produce employment outcomes for individuals with ASD. The interventions needed to address social, behavioral, and/or cognitive dimensions related to the acquisition and maintenance of employment among the study participants were reviewed. The interventions also needed to involve relatively specific and structured experiences designed to support employment placement: for example, providing guidance in completion of applications, resumes, and engaging in interviews; shaping of work skills and appropriate employment setting social skills; employment site supports; designing of jobs/tasks around the expressed needs and desires of participants; teaching of work-related communication skills; or working directly with employers in the structuring of work and work setting features for individuals with ASD.
3.1.2.4 Outcome Measures
Eligible outcomes included subsequent attainment of an employment placement and specific data about the duration and/or retention of that placement must have been provided. Gainful employment included competitive, integrated, or supported employment. Sheltered work or non-integrated work was not considered as an outcome measure for this review. Employment encompassed full or part-time placements. Employment assistance interventions of any length duration were included provided adequate description was provided by the authors of the study. See Appendix A: Potential Job Supports and Interventions Glossary for more information about job supports and interventions.
3.1.2.5 Publication Status
Published and unpublished studies were eligible for inclusion in the evidence pool.
3.1.2.6 Country of Origin and Language of Publication
Studies that were conducted in any country were eligible. We did not exclude studies reported in languages other than English, but we did not specifically search for non-English literature; however, we did search selected international databases. Non-English studies that were retrieved or reviewed required the reviewers to obtain assistance from native speakers.
3.2 SEARCH STRATEGY FOR IDENTIFICATION OF RELEVANT STUDIES
The search strategy used for identification of relevant studies is highlighted below.
3.2.1 Electronic Searches
Studies were identified using electronic search techniques of 28 computerized databases. We consulted database thesauri, where they were available to assure that the universe of appropriate synonyms had been included in the intervention and outcome search term categories. Search terms and search strategies were modified to fit individual databases.
Databases searched included:
PubMed/MEDLINE NARIC REHABDATA ERIC CIRRIE (Center for International Rehabilitation Research Information and Exchange Database) PsycINFO ARD (Autism Research Database) Sage Family Studies Abstracts Dissertation Abstracts International Cochrane Central Registry of Controlled Trials British Education Index Canadian Research Index Australia Education Index CBCA Education Sociological Abstracts Academic Search Complete CINAHL Plus with Full Text Professional Development Collection PsycCRITIQUES PsycEXTRA Psychology and Behavioral Sciences Collection Web of Science Academic One File WorldCat [for monographs] FRANCIS ABI Inform Global Science and Technology Collection PsycARTICLES Business Source Complete
3.2.2 Search Terms
The keywords used in the computerized bibliography searches were divided into three categories: population, treatment, and domain. The searches covered the period from 1943 through 2008. All search terms were truncated using the DIALOG Database conventions in order to include variations in endings of words and in spelling. Terms from the three categories were connected with “or” within each category and connected with “and” between categories.
POPULATION autis? childhood(w)disintegrative(w)disorder? pervasive(w)developmental(w)disorder? pervasive(w)developmental(w)disorder(w)not(w)otherwise(w)specified? spectrum(w)disorder? Asperger? TREATMENT treatment? intervention? model? program? practice? instruction? training? service? supported employ? DOMAIN employ? adult? rehabilitation? vocational? workplace?
For more information on the search strategy, see Appendix B: Documentation of Search Strategies for the Systematic Review.
3.2.3 International Contacts
Our efforts to find studies from outside the United States included searching in several non-United States and international databases. This did yield studies that were reviewed in Stage One review procedures. Recent reviews on autism research outside the U.S. were reviewed to provide additional relevant research studies.
3.2.4 Grey Literature
Grey literature identified through electronic searches was submitted to the same inclusion criteria as other studies. The same time range (1943 - 2008) for grey literature types of studies was specified as the other studies. References from individual studies were searched for potential studies to consider for inclusion. In addition, unpublished dissertations and theses were identified through the search strategy for review and consideration. Also, recent (2010-2011) conference proceedings from relevant associations and conferences were reviewed to identify unpublished studies to include in the review.
3.2.5 Cross-referencing of Bibliographies
The references in relevant journal articles and other reports of research results were scanned for new additions for the review.
3.2.6 Conference Programs
Recent conference programs and conference syntheses were reviewed for leads about eligible literature for review. Professional organizations/events that were reviewed included:
Autism Society of America
National Association of Rehabilitation Research and Training Centers
International Society for Autism Research
National Alliance for Autism Research
Autism Research Institute
National Autistic Society (UK)
Autism Research Centre (Cambridge)
Conference proceedings that were reviewed included:
Asia Pacific Autism Conference 2009
PENN Autism Network Conference.
Autism Society National Conference
NARRTC Annual Conference
Annual International Meeting of Autism Research (IMFAR)
National Autistic Society's Professional Conference
Cambridge Autism Research Conference
3.3 CODING PROCEDURES AND CATEGORIES
Studies were screened for inclusion/exclusion decisions at two stages, Stage 1: citation and abstract and Stage 2: full-text. The same two coders served as independent reviewers at both stages. A third party was not needed to resolve a coding value difference.
3.3.1 Citation and Abstract Stage
At Stage 1, the decision for advancing the retrieved citations and abstracts to the full text stage retrieval was made independently by both reviewers based on meeting two items from a, b, and c of the following questions or a designation by a reviewer of ‘unsure’ (item d):
Are the participants identified, described, and defined under the Autism Spectrum Disorder category? Are the participants 18 years of age or older? Are the participants no longer enrolled in a school-to-work transition program or secondary-level education/program? Is this abstract/citation about employment? Unsure of meeting inclusion criteria?
If the reviewers were ‘unsure’, the citation/abstract was advanced to the Full-Text stage for a final inclusion decision. Inter-rater reliability was tested on a sample of 25 studies at this stage and was found to be 95%. Coding differences were resolved through discussion between the two reviewers.
3.3.2 Full-Text Level
At the Full-Text Stage 2 level, full texts of all citations advanced from Stage 1 were obtained and coded for an inclusion/exclusion decision. The decision for advancing the retrieved full-text studies to an inclusion status was made by two reviewers for each study, independently evaluating each study. An inclusion decision for advancement to the coding stage of the process required that a study met all the criteria presented earlier. Inter-rater reliability was established prior to initiating coding activities, minimizing coding disagreements. When differences did arise, resolution occurred through discussion and agreement of the two reviewers.
At the Full-Text Stage 2 level, the two reviewers recorded all excluded studies and the reason for exclusion independently. For more information see Appendix D: Reasons Stage 2 Studies were Excluded from the Systematic Review.
When multiple studies used the same sample or outcome data, the study providing the most complete information focusing on our desired intervention outcome was selected for inclusion.
3.4 ASSESSMENT OF METHODOLOGICAL QUALITY
Included studies were coded by two independent reviewers for methodological quality addressing dimensions that included:
Design type RCT Individual Randomized Design RCT Group Randomized Design Quasi-Experiment: Equivalent Comparison Design (individuals) Quasi-Experiment: Equivalent Comparison Design (groups) Quasi-Experiment: Nonequivalent Comparison Design (individuals) Quasi-Experiment: Nonequivalent Comparison Design (groups) Quasi-Experiment: Regression Discontinuity Single Group Quasi-Experiment: Interrupted Time Series Design Single Group Pretest/Posttest design ABAB Single Subject Design MBL Single Subject Design Survey: Cross Sectional Survey: Longitudinal Unit of assignment (e.g., individual vs. group/class) Unit of analysis (e.g., Intention to Treat, Test only, Treated) Attrition from pretest to posttest Fidelity of implementation (e.g., following replicable program of intervention) Blinding of assessors/interventionists
Other data that were extracted and coded from the primary studies included: publication source, subject characteristics, sample source, employment setting, intervention characteristics, type of employment, and outcome measurement. See Appendix C for a copy of the coding form.
In addition, an evaluation of the potential risk of bias of all included studies was conducted using “Risk of Bias” procedures developed by Higgins and Green (2011) in which studies are evaluated across five sources of potential bias. A report of this analysis is provided in the Results section.
3.5 CALCULATING EFFECT SIZES
The magnitude of the intervention effect was to be calculated using the commonly accepted statistical formulae and dedicated programs available. The characteristics of the studies included in the final pool, however, did not allow the reviewers to calculate and integrate effect sizes. Specifically, all effect size calculations were conducted using Comprehensive Meta-Analysis (CMA) (Borenstein, Hedges, Higgins, & Rothstein, 2005) using the standardized mean difference statistic d as the measure of treatment effect. Effect sizes were calculated directly from reported means and standard deviations for the experimental and control groups for studies that reported such statistics. For studies that reported statistics such as t, F, or p values and the accompanying sample sizes only, conversion formulae were provided by CMA to calculate the d-index for the effect size estimate. In addition, all effect size intervals were calculated using a 95% confidence interval as provided by the CMA software.
The outcomes reported in the studies included in the final pool, however, did not allow the reviewers to synthesize the effect size estimates due to the lack of common outcomes across the included studies.
4 Results
4.1 INFORMATION RETRIEVAL
The combined electronic and hand searches produced a total of 8,528 citations at Stage 1 Abstract/Citation. Of these studies a total of 77 citations were advanced for collection of a full text copy of the study (Full-Text Stage 2).
Upon review of the full-text for each of the 77 studies, two studies were retained for having met all inclusion criteria described earlier. A list of the excluded studies and the reasons for exclusion are presented in Appendix D. Studies that did not describe or assess an intervention (n = 49), did not present employment-related outcomes (n = 9), did not contain participants with ASD (n = 5), did not meet age or out-of-school criteria (n = 4), did not provide quantitative data (n = 4), or did not include a comparison group (n = 1) were excluded.
One author (M.D. Smith) was contacted regarding several full text studies that were not available to the reviewers. Smith was not able to provide the full text versions but did suggest an alternative study that was accessed and subsequently excluded.
4.2 PUBLICATION BIAS
The authors reviewed unpublished dissertations and theses that were identified through the review's search strategies. In addition, bibliographies of relevant studies were reviewed to identify additional studies that may not have been identified through the database searches. Recent conference proceedings of relevant professional associations and disability-related organizations were reviewed for relevant studies that may not have been published. While these steps added studies to the review process, they did not result in additional included studies.
The authors searched widely for relevant literature. In the authors' opinion, no publication bias exists in the conduct of this systematic review. It is apparent that very limited rigorous research literature currently exists that is directly related to our review question.
4.3 STUDY SUMMARY AND DATA ANALYSIS
In general, the two included studies described the effects of a supported employment approach for adults with ASD on either employment outcomes or aspects of cognitive functioning. The nature of the data provided did not lend itself to a traditional meta-analysis. Since the number of studies was small, and study design and outcomes were diverse, no attempt was made to aggregate results across studies. The following is a descriptive summary of the methodological quality and results of the studies meeting the inclusion criteria.
4.3.1 Methodological Quality Assessment
Methodological quality was assessed by two independent reviewers by evaluating the five elements presented in Table 1. Both of the included studies were quasi-experimental studies for which participant assignment to groups was not random and no assessment of the intervention outcomes was conducted under an examiner or participant blind procedure, and no information was provided regarding the fidelity of the intervention implementation.
ELEMENTS ASSESSING METHODOLOGICAL QUALITY OF INCLUDED STUDIES
The data provided were inconsistent and the authors were unable to verify final participant numbers
In addition, an analysis was conducted by two reviewers of the risk of bias using the five sources of bias shown in Table 2. (Higgins & Green, 2011). A value of “high” suggests that there is substantial potential for the introduction of bias for the particular dimension represented. This analysis suggests a judgment of potentially high risk of bias for three of the five sources of risk bias assessed.
RISK OF BIAS RATING FOR INCLUDED STUDIES
As reported in the following study results, there were no common outcome measures reported across the two studies and all effect size estimates were based on a single measurement. With these shortcomings in participant selection, performance, detection, attrition, and reporting characteristics the potential risk of bias was judged to be high thus, reducing the potential impact of the findings. It should be noted that even though the exact attrition value associated with the Garcia-Villamisar et al. study could not be determined, the maximum attrition rate possible in the reported data was less than 10% and is consistent with the classification of low risk of bias.
These data taken as a whole suggest that the overall quality of the evidence available for assessing the impact of programs to improve the employment conditions of adults with autism is weak.
4.3.2 Study Results
Two studies, Mawhood and Howlin (1999) and Garcia-Villamisar, Ross and Wehman (2000) were coded as quasi-experimental (QED) studies. See Table 3 below for more information about the two included studies. Mawhood and Howlin described outcomes directly related to employment including employment rate, duration of job retention, and earned wages by comparing an experimental group that consisted of 30 individuals (27 males, 3 females) with a control group consisting of 20 males all seeking employment. The experimental group received guidance from a support worker in the form of job finding, work preparation, and communication with the employers while the control group did not receive any support. Over the two-year period, more participants from the supported group when compared to the control group (a) found paid employment than did participants from the control group (d = 1.067, 95% CI = 0.123 to 2.010) (b) did not work significantly greater number of hours per week (d = 0.328, 95% CI = -0.628 to 1.284), (c) were employed for a longer period of time (d = 0.684, 95% CI = 0.111 to 1.257) and (c) made significantly higher wages (d = 1.177, 95% CI = 0.169 to 0.819). In addition to the job related outcomes, Mawhood and Howlin also assessed participant self esteem and found no significant difference between the experimental and control group participants (d = -0.037, 95% CI = -0.594 to 0.520).
Garcia-Villamisar, Ross, and Wehman (2000) described effects of a supported employment program on aspects of clinical symptomatology for adults with autism. They examined the differential impact of supported versus sheltered work for adult participants with autism. The supported group had jobs located in the community, worked between 15 and 30 hours, and received job coaches. The sheltered work group did not have jobs in the community and did not receive job coaches.
At the beginning of the study, there were no statistically significant differences between scores on the Childhood Autism Rating Scale (CARS), after three years, the supported group's performance on the CARS was not statistically different from the sheltered group, (d = -0.229, 95% CI = -0.764 to 0.306). Garcia-Villamisar et al. (2000) did not report any specific gains or maintenance of employment; however, they argued that individuals with high symptomatology, or challenging characteristics and behaviors, can be unsuitable candidates for competitive employment. Because they found that participation in supported employment compared to sheltered work prevented deterioration in autistic pathology, one can infer that supported employment contributed to greater employment maintenance.
Multiple attempts to correspond with Dr. Domingo Garcia-Villamisar in Spain in an effort to obtain additional information about sample characteristics in both included and excluded studies he authored were unsuccessful. Specifically, Garcia-Villamisar et al. reported inconsistent sample sizes throughout four separate instances for the overall sample and the supported and sheltered employment groups: 1) overall sample: 51 participants, supported employment: 25 participants, sheltered work: 26 participants; 2) overall sample: 55 participants, supported employment: 25 participants, sheltered work: 26 participants; 3) supported employment: 21 participants; 4) Overall sample: 53 participants, sheltered work: 29 participants, supported employment: 24 participants. For analysis purposes, reviewers used the most conservative numerical values in order to produce the most conservative effect size value for this study.
5 Discussion
This systematic review sought to determine the effectiveness of adult employment assistance services for persons with autism spectrum disorders (ASD). The available data for drawing a ‘what works’ conclusion are minimal. The scientific quality of the evidence is weak, and based on the number of citations identified at the initial stage of the informational retrieval process, there is certainly a need for a more rigorous and controlled study of the effectiveness of employment programs for individuals with ASD. With only two studies available for inclusion and both of them representing a research design that precludes a substantive causal effect interpretation, any conclusions drawn must be viewed with caution.
In terms of the benefits of participation in a program of supported employment, both Mawhood and Howlin (1999) and Garcia-Villamisar et al. (2000) demonstrated an overall advantage of either no intervention or a supported employment advantage over a sheltered workshop condition. However, because none of the outcomes of the two studies were similar, no syntheses of these data were appropriate. No other experimental or quasi-experimental studies were located for this review that would allow for an assessment of the impact of employment training for adults with ASD on the acquisition or maintenance of employment. Given the relatively small number of participants, no judgment can be offered as to the efficacy of employment training for individuals with ASD that would be applicable to a large-scale application.
As questions about employment options for persons with ASD increase, it is significant to note that several authors (Dew & Alan, 2007; Holmes, 2007; Hurlbutt & Chalmers, 2004) have indicated that individuals with ASD are more likely to lose employment due to a difficulty involving social interaction/behavior than they are to lose employment due to an inability to perform their assigned job skills. Holmes (2007), for example, indicated that a major reason for underemployment, unemployment, and job loss of individuals with ASD is the failure to determine and provide the supports needed. The need for more research to definitively identify what works for individuals across the autism spectrum is evident to help support both the design of appropriate and effective support services and their widespread availability at the community level.
It may be that individuals with ASD need extensive training or instruction in order to acclimate to the work environment and to develop the necessary skills needed for the job. Employment training programs that provide individualized job training and support for placement and sustained employment are of interest to the ASD community. Future research is needed to assess the efficacy and effectiveness of such programs, the components of those programs, and the benefits of those programs to individuals with ASD, their family, the employer, and society.
5.1 LIMITATIONS OF STUDIES
The primary limitation of this review is the lack of a sufficient number of appropriately designed studies that would allow for a causal interpretation of measured outcomes based on individuals with ASD participating in employment programs. The small number of studies resulted in a shortage of compatible and relevant data to conduct substantial analyses.
5.2 IMPLICATIONS OF ASSOCIATED QUALITATIVE RESEARCH AND OTHER RELATED STUDIES
This systematic review originated with a question focusing on identifying effective interventions to support persons with ASD in acquiring and sustaining employment. In the course of the review process, a significant body of qualitative literature was identified that augments the findings from the two quantitative studies included in the review. A summary of the implications of these to the findings of this review are described here in terms of their relationship to the employment of people with ASD. Areas with implications of note include:
Identification of most appropriate work settings and placements Provision of effective supports on the job Need for long-term support services for the employer and consumer Costs of support Effects of employment on persons with ASD
5.2.1 Identification of the Most Appropriate Work Settings and Placements
Appropriate work settings are important to all employed people, including those with ASD. Individuals with ASD choose a wide variety of careers reflecting a diversity of vocational interests. Supported by the qualitative literature, work placements based on an individual's strengths were crucial to sustained competitive employment (Keel, Mesibov, & Woods, 1997; Muller, Schuler, Burton, & Yates, 2003; Smith, Belcher, Jurhs, & Nabors 1994). Although there was a variety of specific jobs where people with ASD work, commonalities emerged across appropriate work settings, including flexible work schedules, low social interaction, clear expectations of duties, low levels of sensory simulation, and built-in time to learn new skills (Keel et al., 1997; Muller et al., 2003; Smith et al., 1994; Wehman & Kregel, 1988). Additionally, appropriate work settings for people with ASD ideally included those that ensure employers and colleagues are knowledgeable about the autism spectrum (Keel et al. 1997).
5.2.2 Provision of Effective Supports on the Job
A common theme noted among the qualitative studies reviewed was the importance of job coaching. Specifically, the use of behavioral techniques such as functional behavioral assessment, response cost procedure, positive reinforcement, social skills training, prompt fading, task analyses, and task preference assessments were reported as effective job supports (Burt, Fuller, & Lewis, 1991; Muller et al., 2003; Smith & Coleman, 1986; Wehman & Kregel, 1988). Natural supports, such as the use of co-workers to provide on-site training, social skills training, or transportation, have also been found to be effective in reducing the amount of training provided by the employment specialist (West, Kregel, Hernandez, & Hock, 1997). Furthermore, participants who received simulation training in addition to on-the job training acquired the necessary job skills more quickly, thereby decreasing the amount of training required (Lattimore, Parsons, & Reid, 2006). Equally important, employment supports including job finding and tailored job placement were positively related to successful placement in competitive employment settings of individuals with ASD (Schaller & Yang, 2005).
The qualitative literature suggested that the following vocational supports were most important for success for employees with ASD: appropriate tailored job matching, individualized ASD-specific job supports, social communication supports, autism awareness training, and employer and employee-focused attitudinal supports (Muller et al., 2003).
5.2.3 Need for Long-term Support Services for the Employer and Consumer
The identified qualitative literature also suggested that long-term vocational support services were critical for sustaining employment and should involve both employees and employers (Wehman & Kregel, 1988). Though short-term vocational supports were important in helping individuals with ASD learn how to perform work tasks, long-term supports were needed for people with ASD to sustain employment and adapt to changes in the work environment (Nesbitt, 2000; Schaller & Yang, 2005). An employee's sense of predictability was helpful for daily successful work performance. Furthermore, when changes occurred such as a new staff member, new supervisor, transportation issues, or other life changes occurred, vocational support services facilitated the employee's adaptation to the change (Keel et al., 1997; Schaller & Yang, 2005).
5.2.4 Costs of Support
A common issue reported in the qualitative literature with respect to successful long-term support was the funding of support services. Indeed, the benefits of funding such support services include the increase in employees' income that comes with sustained employment, a decrease in the need for public benefits, and a decrease in the day-to-day living costs shouldered by family members (Howlin, Alcock, & Burkin, 2005). It was noted that many adults with ASD received care at alternative day programs. Community-based competitive employment-oriented vocational support programs, like those we describe in this review, typically cost more than other employment alternatives such as sheltered workshops. However, Howlin et al. (2005) showed that the gap is narrowing between the costs of these support programs. In fact, Cimera and Cowan (2009) show that the state provided vocational rehabilitation services for persons with ASD are expensive but becoming less so in the four year period from 2002 through 2006. In time, costs may reach a “break-even” point eliminating cost differentials between facility-based and community-integrated alternatives. In fact, Cimera (2010) has calculated the benefit cost ratio of persons served by state vocational rehabilitation agencies in the U.S. from the period 2002 to 2007. He found that those placed in supported employment generated an average monthly net benefit of $475.35 and a benefit cost ratio of 4.20. It seems clear, that significant benefits in terms of social integration, higher wages, and community involvement accrue to persons with ASD in competitive, integrated employment settings.
5.2.5 Effects of Employment on Persons with ASD
Another common theme emerging from the qualitative literature was that employment produced beneficial effects in the behavior and environment of persons with ASD. Employees with ASD and longer lengths of employment tended to be more successfully integrated into the culture of their work place. For example, employees with ASD who had been employed longer tended to have regular contact with co-workers, share work areas, and interact socially with employees without disabilities who had been employed for shorter periods of time. Additionally, after a period of six-months of employment, improvements in areas of independent functioning, expression, domestic and socialization skills were observed (Smith & Belcher, 1994; Wehman & Kregel, 1988).
5.2.6 Additional Detail Across Studies
Table 3: Characteristics of Studies Cited gives additional detail about the included and excluded studies cited in this Discussion section. The table cites the study, project design, participants, duration, program description, and outcome areas for each study and allows for comparison across these studies.
CHARACTERISTICS OF STUDIES CITED
Employment status was not a dependent variable in Muller et al. (2003).
5.3 IMPLICATIONS FOR RESEARCH
This review demonstrates the lack of available research that addresses the efficacy of a variety of employment interventions supporting individuals with ASD. Future research efforts are needed in the development of (1) group randomized controlled trials (RCT) or quasi-randomized designs (QED) that utilized well controlled pre-intervention selection and assignment, and (2) outcome measures for both job specific skills and related skills that have a significant impact on obtaining and sustaining meaningful employment in society at large.
The construction of studies involving individuals with ASD is generally accepted to be difficult at best due to the low incidence and the heterogeneity of the population; thus, we found few individual studies with sufficient statistical power to answer questions of intervention efficacy. The authors of this review would argue that future research needs to focus on the production of studies using small “n” group samples utilizing as rigorous a scientific design as possible. That is, multiple small “n” group studies, over time, could provide a “synthetic n” large enough to conduct a meta-analysis that would yield results useful for wider application across the spectrum of ASD. These studies would focus on the critical elements of employment training for adults with ASD such as:
Program Characteristics (e.g., employment support models) Social Skills Training Customization of Employment Placement Job-Skill/Interest/Motivation Matching Financial Impact of Employment Training Employment Sustainability
5.4 PLANS FOR UPDATING THE REVIEW
The authors will examine the review every three years after publication for update as per C2 policy.
Acknowledgements
The concept for this systematic review was included in a proposal submitted to and funded by the National Institute on Disability and Rehabilitation Research in the Office of Special Education and Rehabilitative Services in the U.S. Department of Education.