A Systematic Review of Assistive Technology for Individuals With Intellectual Disability in the Workplace

Abstract

Research has shown that assistive technology (AT) can help support employment skills for individuals with intellectual disabilities (IDs). The authors of this review examined the effectiveness of various types of AT support for individuals with ID in the workplace with a focus on the participant’s independent ability to demonstrate a specific employment skill. Ten articles published between 2006 and 2016 met inclusionary criteria set forth for this review. Overall, AT support resulted in medium to highly effective effect size gains (PND 91%; standardized mean difference 2.84; Tau-U 0.902), indicating effects were generally meaningful. Findings from these studies indicate AT was successful for increasing work performance of individuals with ID in respect to productivity, navigation, time management, and task completion. This review also provides an analysis of findings, discussion on limitations, and recommendations for future research.

Keywords

assistive technology technology perspectives intellectual disability exceptionality employment

Employment is one of the several important factors critical to increasing the quality of life of individuals with intellectual disabilities (IDs; Nota, Ginevra, & Carrieri, 2010; Schalock et al., 2002). Unfortunately, the employment rate for individuals with ID (38.8%) is only about half that of the general population (Newman et al., 2011). Individuals with ID often experience a combination of both intellectual and adaptive functioning deficits that impact conceptual, social, and practical domains (American Association on Intellectual and Developmental Disabilities, 2013). These deficits can negatively impact an individual’s ability to gain and sustain employment due to limited cognitive abilities, which inhibits their working memory and conceptual understanding (Davies, Stock, & Wehmeyer, 2002). Individuals with ID often experience difficulty concentrating, poor communication skills, health problems, and overall lack of motivation (Rose, Saunders, Hensel, & Kroese, 2005). In addition, deficits in time management skills, self-regulation, organizational skills, and the ability to execute tasks can limit these individuals’ success in gaining and maintaining employment (Davies et al., 2002; Green, Hughes, & Ryan, 2011; Smith, Polloway, Smith, & Patton, 2007).

Given these skill deficits, individuals with ID often require continued assistance, training, and frequent prompts to increase job skills and employability (Sauer, Parks, & Heyn, 2010). To date, the success of many individuals with ID in vocational settings has resulted from the combination of supported employment (Citron et al., 2008) and/or job coaches (Wehman, 2011). However, this continued dependency upon others can lower an individual’s self-esteem, which can negatively impact their quality of life. Hence, it is critical for individuals with ID to shift stimulus control from a teacher or job coach to some other stimulus within the natural environment to better promote independence in an employment setting (Beyer & Perry, 2013). One method that has shown tremendous promise for increasing independent task performance and improving the efficacy of task completion skills for individuals with ID is assistive technology (AT) support (Davies, Stock, & Wehmeyer, 2003; Gilson, Carter, & Biggs, 2017; Wehmeyer et al., 2006).

Types of AT Support

AT is defined as any object or system that is acquired commercially, modified, or customized to increase or maintain the capabilities of people with disabilities (Individuals with Disabilities Education Improvement Act, 2004). AT support can take the place of human assistance to promote independence in the workplace (Cihak, Kessler, & Alberto, 2008). Prompts provided by AT devices have been shown to help individuals complete assigned tasks independently (Chang, Chou, Wang, & Chen, 2013). Various types of AT prompting systems are emerging to promote skill acquisition and task performance. To date, studies have used a number of different types of AT to improve work performance for individuals with ID, including handheld computers, wearable technology, and portable electronic devices. A brief overview of each is provided.

Handheld computers

Handheld computers (e.g., Axium X30 model and HP iPAQ pocket PC) are commonly described as portable devices with a small keyboard to input information that are capable of being completely functional while holding it. Due to their compact size, these computers can be stored in a pocket and are sometimes referred to as a personal digital assistant. These computers have the potential to assist individuals with ID by enhancing their organizational abilities and enabling them to complete work independently. These devices promote a shift in stimulus control to improve employee autonomy and decrease dependency on teachers, job coaches, coworkers, and peers (Cihak et al., 2008; Ferguson, Myles-Smith, & Hagiwara, 2005; Furniss et al., 2001; Riffel et al., 2005). Several researchers have prototyped handheld prompting systems to support individuals with ID in vocational settings by integrating laptops and special purpose communicators with AT to provide task prompting (Cannella-Malone et al., 2006; Van Laarhoven & Van Laarhoven-Myers, 2006).

Wearable technology

Wearable technology is a more recent commercially available innovation in technology that incorporates smart sensors. The purpose of wearable technology is to provide constant, portable, and mainly hands-free digital access. The smartwatch is an example of a modern wearable technology that can benefit people with ID (Fichten, Asuncion, & Scapin, 2014). Smartwatches exchange data with other connected devices and operators without requiring any human intervention, thus allowing users to independently access a multitude of apps to support productivity as well as the ability for employers to monitor efficiency. Vibrating watches are another simple form of wearable AT that require minimal skills but are highly effective in providing notifications to the wearer regarding daily events and routines (Green et al., 2011). These technologies can provide numerous supports that might otherwise require an assistant to either monitor or prompt an individual to complete a work-related task (Mechling, 2011).

Portable electronic devices

Portable electronic devices (e.g., iPod and iPhone) are electrically powered light weight portable devices that can store, process, and transmit information to an individual. In addition to portability, these devices have the capability of providing different types of prompts to assist individuals with ID complete a task instead of relying upon human support (Douglas & Uphold, 2014). Portable electronic devices have been successfully used to assist individuals with ID perform work-related tasks independently by providing users with audio, video, and picture prompts (Davies et al., 2002; Riffel et al., 2005; Van Laarhoven, Van Laarhoven-Myers, & Zurita, 2007).

Purpose

Previous reviews have shown AT has the potential to help individuals with ID overcome many limitations and provide the necessary accommodations to improve the completion of work-related skills (Sauer et al., 2010). Given the rapid pace of improvement in technology, recent application development, and increased employment outcomes due to AT, there is a need to update research findings on work-related tasks utilizing AT for individuals with ID. Hence, the purpose of this review is to investigate the efficacy of AT support in improving the ability to complete work-related tasks independently and efficiently as well as other employment skills for individuals with ID. Emphasis is placed on identifying the efficiency of varying types of AT, gaps in the current research, and identifying possibilities for future research. Specific research questions addressed include: (a) What types of AT have been assessed for individuals with ID in the workplace? (b) What types of work skills do AT assist individuals with ID in employment setting? and (c) What is the efficacy of AT for individuals with ID in employment settings?

Method

Literature Search Procedures

A comprehensive systematic search was conducted in accordance with preferred reporting items for systematic reviews and meta-analysis standards (Liberati et al., 2009) in an effort to locate all relevant research studies targeting the impact of AT in improving performance of work-related tasks independently and efficiently for individuals with ID. The following procedures were used to locate articles through a multiple step process, including an electronic search, hand search, ancestral review, and forward search.

Eligibility Criteria

Inclusionary criteria for this review required studies (a) involve the implementation of AT as an independent variable, (b) examine the participant’s independent ability to demonstrate a specific employment skill as a dependent variable, (c) include participants identified as having an ID, (d) use an experimental design, and (e) be published in peer-reviewed journals between 2006 and 2016. See Figure 1 for details of search results that resulted in identifying 10 articles for inclusion in this review. The most common reasons for article exclusion included studies with (a) no relation to employment skills and (b) participants not having an ID.

Figure 1.

Preferred reporting items for systematic reviews and meta-analysis flowchart.

Electronic search

First, an electronic search was completed in September 2016 using the following electronic databases: Business Source Complete, Education Resources Information Center (ERIC), PsychInfo, Education Research Complete, EconLit, Education Full Text, SocIndex, and Social Sciences Full Text. Parameters of the search included peer-reviewed articles from 2006 to 2016. The search was conducted over the past 10 years to ensure inclusion of relatively current technology. Search terms included: (assistive technology) OR (assistive devices) OR (computer application) OR (apps), OR (iPad), OR (iPhone), OR (tablet), OR (handheld), AND (intellectual dis*), OR (mental retardation), OR (development*dis*), OR (cognitive dis*), OR (cognitive impairment), AND (work), OR (workforce), OR (employment), OR (job), OR (workplace), OR (vocation).

Hand search

Next, a hand search was also completed on two journals (Journal of Special Education Technology and Research in Developmental Disabilities) which published the most articles that were located through the electronic search. The hand search revealed one additional study that met the inclusion criteria.

Ancestral review

Ancestral reviews were conducted with articles referenced in literature reviews as well as the reference sections of all articles that met the criteria for inclusion of this review. No additional articles meeting inclusion criteria were located.

Forward search

Finally, a forward search was conducted by entering studies that met inclusion criteria into the Web of Science database to locate other relevant works that cited each of the accepted articles. No additional research articles were found to consider for inclusion.

Data Collection

Coding procedures

Operational definitions and a coding form were developed to record information contained in each article regarding participants, setting, design features, quality indicators (QIs), and outcomes. The following items were used in the collection of the data, including sample size, age of participants, gender, race, IQ, diagnosis, placement setting, treatment setting, and treatment group size. Other items coded included research design, dependent variable, treatment focus, type of application or software used, and type of device used. Additional items also focused on the intervention length and time per session, social validity, fidelity procedure, effect size (ES), significant differences, and group means. A copy of the coding sheet can be obtained through the first author. Two graduate students met and compared each coding and calculated reliability as the percentage of agreement. Initial inclusion reliability was 92–100%. After consultation, consensus was met resulting in 100% agreement for inclusion of articles.

QIs

Following recommendations set forth by Cook and colleagues (2015), the 10 articles that met initial inclusion criteria were evaluated by two graduate students to determine the number of Council for Exceptional Children (CEC) standards for evidence-based practices in special education that each study met. The intent of CEC’s QIs is to determine which studies have the minimal methodological features to merit confidence in their findings, which entails providing information regarding (1) context and setting, (2) participants, (3) intervention agent, (4) description of practice, (5) implementation fidelity, (6) internal validity, (7) outcome measures, and (8) data analysis. According to the CEC (2014), methodologically sound studies must meet all of the QIs identified. Reliability of coding for CEC standards ranged from 95% to 100% agreement, with a mean of 98% agreement. Any disagreements in coding were resolved by reaching consensus through discussion.

Data extraction

Data were extracted from each study’s graph using DigitizeIt 2.2 software. This software has the ability to extract data values from graphs presented in scientific publications where only charts and graphs are given with no data values. Often data are presented in X–Y scatter or line plots and in order to use the data, it must be digitized. This software provides the ability to take a scanned image of a plot and quickly digitize values off the plot by clicking on each data point. This offers an efficient way to retrieve the numbers from the plots provided. When evaluating studies that involve single-case research design, it is recommended that multiple methods be used to increase confidence in findings (Gast, 2005). Given the current limitations regarding ES methodologies, multiple outcome measures were calculated including several of the more popular historical methodologies for comparative purposes as well as more advanced methods. These include visual analysis, percentage of nonoverlapping data (PND) points, standardized mean difference (SMD), and Tau-U.

Data Analysis

Visual analysis

While there has been much dispute regarding the best method to calculate an effect of single-case design (SCD), there is general agreement that the primary method of assessment has been and should remain visual analysis (Rakap, 2015; Wolery, Busick, Reichow, & Barton, 2010). Visual analysis provides researchers conducting an SCD critical information regarding (a) pattern of baseline performance, (b) when to initiate or terminate a phase, (c) whether the treatment requires adjustment to improve outcomes, and (d) if a participant is responding to an intervention (Shadish, Hedges, Horner, & Odom, 2015). Visual analysis provides a useful tool for making a summative judgment of the outcome of treatment for a case (Perone, 1993). Visual analysis of all graphed data was completed for each single case study in accordance with Kratochwill et al. (2010). This technique allowed for analysis of changes in level, trend, variability, immediacy of the effect, overlap, and consistency of data patterns across similar phases. The senior author was trained in these procedures through an Institute of Education Sciences (IES) sponsored training seminar.

Percentage of nonoverlapping data (PND)

PND was also calculated given it is the most frequently used measure of ES in single-case research (Scruggs & Mastropieri, 2013) and can be applied to the majority of single-case research designs (Kavale, Mathur, Forness, Quinn, & Rutherford, 2000). It also has the advantage of correlating with visual analysis (Parker, Vannest, & Davis, 2011). PND was calculated by tallying the number of intervention data points that exceed the maximum baseline data point and dividing by the total number of intervention data points. To interpret, PND values between 91 and 100 are considered highly effective, between 71 and 90 are moderately effective, between 51 and 70 are mildly effective, and between 0 and 50 are ineffective (Scruggs & Mastropieri, 2013).

SMD

SMD was first introduced over 30 years ago to compare results across single case studies that used different measures (Gingerich, 1984) and has since been commonly used within behavioral and social science research. SMD were calculated using Busk and Serlin’s (2005) formula of subtracting the mean of the baseline phase from the mean of the intervention phase and dividing by the standard deviation of the baseline. Interpretation of SMD follows that of Cohen’s (1988), where 0.20 is considered a small effect, 0.50 is a medium effect, and 0.80 is considered a large effect. ES’s for individual participants or dependent variables that were statistical outliers (i.e., ESs > 3.0) were capped at 3.0 to prevent them from disproportionately affecting the outcomes of the studies when grouped.

Tau-U

Tau-U has become one of the more commonly used measures of ES today due to its nonparametric approach. Tau-U is derived from Kendall’s and Mann–Whitney U (see Parker, Vannest, Davis, & Sauber, 2011) and is calculated by merging trend and nonoverlap data. Tau-U provides a calculation of both confidence intervals and p values (Soares, Harrison, & Vannest, 2016). Advantages of Tau-U include its (a) consistency with visual analysis, (b) applicability for short data series and simple designs, (c) appropriateness with any type of single case research design (SCRD), (d) strong statistical power, (e) control for trend during baseline phase, (f) usefulness with nonaggregated data from either a single client or a complex design, and (h) usefulness for meta-analysis (Parker & Vannest, 2012; Parker et al., 2011). Tau-U was calculated using the online Tau-U calculator (Vannest, Parker, & Gonen, 2011) for each individual contrast between the baseline and adjacent intervention contrast for each academic measure. Tau-U effects are measured as small (0–0.65), medium (0.66–0.92), and large (0.93–1.00), which are equivalent to ranges recommended for nonoverlap of all pairs (Parker, Vannest, & Brown, 2009).

Results

Study Selection

In all, 10 articles met our inclusion criteria of studies related to AT with ID. Coders came to an agreement on 10 of the articles used for review. The percentage of agreement ranged from 92% to 100%, with a mean of 96% agreement. The coders came to a consensus on the remaining through discussion for entry into the analysis.

CEC QIs and Study Characteristics

Study characteristics for the 10 articles are summarized in Table 1. An analysis of the 10 single case studies using the eight CEC (2014) QIs are provided in Table 2.

Table 1.

Study Characteristics of Individuals With Intellectual Disabilities in Employment Settings.

Study	Participant N (Age Range)	Dependent Variable	Type of AT (App/Software)	Treatment Setting and Task
Cihak, Kessler, and Alberto (2007)	n = 4 (18–19 years)	Using prompting system to determine percentage of steps performed independently across increasing task complexity	Axium X30 handheld computer—picture perfect software	Stores and restaurant (e.g., self-preparation, gathering carts, stocking milk)
Van Laarhoven, Van Laarhoven-Myers, and Zurita (2007)	n =1 (18 years)	Effectiveness of video-based strategies on independent responding/percentage of prompts	Microsoft Pocket PC 2003 second edition software	Restaurant (recipes, clocking in/out, and cleaning/sanitizing)
Cihak, Kessler, and Alberto (2008)	n = 4 (16–17 years)	Determine efficacy of prompting system and independent transition to a separate vocational task	Axium X30 handheld computer—picture perfect software	Chain of tasks in grocery store, department store, and restaurant (e.g., gathering carts, stocking milk, vacuuming, and preparing rolls)
Van Laarhoven, Johnson, Van Laarhoven-Myers, Grider, and Grider (2009)	n = 1 (17 years)	Effectiveness of prompting devices on independent responding, reduction in prompting for both error correction and technology usage	Video iPod—Pinnacle Studio 10 (2005) software	Animal shelter (cleaning bathrooms, mopping floors, cleaning kernels, and taking out garbage)
Chang, Chen, and Lu (2011)	n = 2 (19–20 years)	Effectiveness of system in reducing number of breaks in performance	G-sensor system installed on smartphone	Assembly workshop in community-based rehabilitation (CBR) program (waste disposal)
Green, Hughes, and Ryan (2011)	n = 1 (22 years)	Effectiveness of AT on timeliness of student	Vibrating watch—calendar clock with time and date, vibration alarm apps	Administrative tasks in university library (organizing, checking, and reshelving books)
Collins, Ryan, Katsiyannis, Yell, and Barrett (2014)	n =3 (21–23 years)	Effectiveness of AT in completion of novel office task	iPod touch—functional planning system app	Office setting (copying, faxing, and scanning)
Douglas & Uphold (2014)	n = 5 (15–16 years)	Effectiveness of ePAS on percentage of tasks independently and correctly completed/percentage of steps correctly completed when programming the device	iPad and iPod touch—first then visual schedule app	School cafeteria (classroom tasks, lunchroom tasks)
McMahon, Cihak, and Wright (2015)	n = 4 (20–24 years)	Effectiveness of navigation tool on percentage of correct independent navigation decision	iPhone—Layar mobile app and Google maps	Community setting (navigation to workplace)
McMahon et al. (2015)	n = 6 (18–24 years)	Effectiveness of navigation tool percentage of independent direction checks	Google maps app on mobile devices and AR navigation app	Navigation in public university campus

Note. n = number of sample; ePAS = electronic photographic activity schedule; AR = augmented reality navigation tool; AT = assistive technology.

Table 2.

Analyses of Studies by Council for Exceptional Children QI.

Intervention Study	QIs								Total QIs Met (%)
Intervention Study	QI-1	QI-2	QI-3	QI-4	QI-5	QI-6	QI-7	QI-8	Total QIs Met (%)
Cihak, Kessler, and Alberto (2007)	^a1/1	^a2/2	^a2/2	^a2/2	2/3	^a6/6	4/5	^a1/1	6/8 (75%)
Van Laarhoven, Van Laarhoven-Myers, and Zurita (2007)	^a1/1	^a2/2	1/2	^a2/2	1/3	^a6/6	^a5/5	^a1/1	6/8 (75%)
Cihak, Kessler, and Alberto (2008)	^a1/1	^a2/2	^a2/2	^a2/2	2/3	^a6/6	4/5	^a1/1	6/8 (75%)
Van Laarhoven, Johnson, Van Laarhoven-Myers, Grider, and Grider (2009)	^a1/1	^a2/2	^a2/2	^a2/2	1/3	5/6	4/5	^a1/1	5/8 (63%)
Chang, Chen, and Lu (2011)	^a1/1	1/2	1/2	^a2/2	0/3	^a6/6	3/5	^a1/1	4/8 (50%)
Green, Hughes, and Ryan (2011)	^a1/1	^a2/2	1/2	^a2/2	^a3/3	^a6/6	^a5/5	^a1/1	7/8 (87%)
Collins, Ryan, Katsiyannis, Yell, and Barrett (2014)	^a1/1	^a2/2	^a2/2	^a2/2	^a3/3	^a6/6	^a5/5	^a1/1	8/8 (100%)
Douglas & Uphold (2014)	^a1/1	^a2/2	^a2/2	^a2/2	^a3/3	^a6/6	^a5/5	^a1/1	8/8 (100%)
McMahon, Cihak, & Wright (2015)	^a1/1	^a2/2	1/2	^a2/2	^a3/3	^a6/6	4/5	^a1/1	6/8 (75%)
McMahon et al. (2015)	^a1/1	^a2/2	1/2	^a2/2	^a3/3	^a6/6	^a5/5	^a1/1	7/8 (87%)
QI met (%)	^b10/10 (100%)	9/10 (90%)	5/10 (50%)	^b10/10 (100%)	5/10 (50%)	9/10 (90%)	5/10 (50%)	^b10/10 (100%)

Note. Each element for the indicator met was scored as 1. Boldface indicates studies met all quality indicators. QI = quality indicators.

^aIndicates all elements in the indicator were met. ^bIndicates all the studies met that indicator.

Context and setting (QI-1)

All 10 studies (100%) provided sufficient information regarding the critical features of the context or setting. The studies were conducted across a variety of settings. Three (30%) studies were performed in special education classrooms (resource/self-contained), three (30%) studies were in a general education classroom, and four (40%) studies were conducted in postsecondary education programs.

Participants (QI-2)

Nine (90%) of the studies provided sufficient information regarding participants (QI-2). The criteria were considered met if the authors provided sufficient information to identify the population of participants, described the demographics relevant to the review, and described disability or risk status of the participants. Studies reported determination of disability status using Individuals with Disabilities Education Act (IDEA) criteria and standard intelligence tests (Wechsler Abbreviated Scale of Intelligence, Wechsler Intelligence Scale for Children third and fourth edition, Stanford Binet fourth edition, Reynolds Intellectual Assessment Scale). One study (Chang, Chen, & Lu, 2011) did not describe disability or risk status of the participants in their study.

Of the 31 students included in all the studies, 19 (61%) were males and 12 (39%) were females. Only three (10%) of the participants’ race were reported. IQ scores were reported for 29 (94%) of the participants. While IQ was not reported for the remaining two (6%) participants, the authors did report they experienced cognitive impairment and were diagnosed with brain injury and paranoid schizophrenia/dementia. IQ scores of participants ranged from 36 to 65, with 13 (42%) falling within the mild (50–70 IQ) range, while 16 (52%) of the participants were in the moderate (35–49 IQ) range.

Intervention agent (QI-3)

This QI was considered met if the studies provided sufficient information regarding the role of the intervention agent and described specific training and qualifications required to implement the intervention. Only five (50%) of the studies met this indicator. Across all studies, professional background information and description of specific training or certification were limited or not reported.

Intervention procedures (QI-4)

All (100%) studies provided sufficient information describing (a) intervention procedures, (b) intervention agents action, and (c) materials provided. This ensured the practice was clearly understood and can be reasonably replicated. As shown in Table 1, varied types of handheld systems (n = 4, 40%), portable electronic devices (n = 5, 50%), and wearable technologies (n = 1, 10%) were used, including iPod, iPhone, iPad, handheld computer, vibrating watch, smartphone, and pocket PC utilizing different software applications. All (100%) devices were used as prompting systems to independently perform tasks related to employment. Prompts across studies included video, audio, gestural, direct verbal, picture, tactile/vibratory, or sometimes a combination of two types of prompts (e.g., audio and text). Seven (70%) of the studies used AT to perform work-related tasks. Two (20%) of the studies were used to make navigation decisions to the workplace, and one (10%) study used AT as an intervention to measure time management skills.

Implementation fidelity (QI-5)

Data on fidelity were evidenced in five (50%) of the studies. To meet this criteria, studies needed to assess and report implementation fidelity (a) related to adherence using direct, reliable measures; (b) related to dosage or exposure using direct, reliable measures; and (c) regularly throughout implementation of the intervention. Fidelity measures included adherence to (a) correct verbal instructions, (b) timely initiation of prompts, (c) error correction procedures, and (d) prompting hierarchy. Fidelity was calculated by dividing the number of steps that the implementer completed correctly by the sum of correct and incorrect steps, then multiplying by 100. Fidelity range across studies were 97–100%. The most common form of fidelity included recorded observations (n = 3, 30%) and itemized checklists (n = 2, 20%).

Internal validity (QI-6)

Nine (90%) of the studies properly addressed internal validity. This indicator was considered met if (a) the independent variable was under the control of the experimenter, (b) the study described baseline and intervention, (c) sufficient evidence was provided that the independent variable caused change in the dependent variable, (d) at least three demonstration of experimental effects at three different times were provided, (e) all baseline included at least three data points, and (f) controls for common threats to internal validity (e.g., diffusion of treatment) were explained. Nine (90%) of the studies had three or more baseline data points. Van Laarhoven, Johnson, Van Laarhoven-Myers, Grider, and Grider (2009) did not collect sufficient baseline points for two of the three tasks with only 1–3 points. Researchers across all studies controlled the manipulation of the intervention.

All 10 (100%) studies reviewed measured the effectiveness of AT tools on employment outcomes through the use of a single case research design (SCRD). The most common SCRD was multiple baseline/probe design which was used in four (40%) of the studies. The next most common designs included withdrawal (ABAB) design used in three (30%) of the studies and alternating treatment design that was also used for three (30%) of the studies.

Outcome measures/dependent variables (QI-7)

Six (60%) of the studies met the QI-7. The indicator was considered met if (a) outcomes were socially important; (b) measurement of the dependent variables was defined and described; (c) effects of the intervention on all measures of the outcome were reported; (d) frequency and timing of the outcome measures were appropriate; and (e) evidence of adequate internal reliability, interobserver reliability, and test–retest reliability was provided. All 10 (100%) studies were related to AT interventions in the workplace, but authors measured a variety of outcome measures as reported in Table 1. All 10 (100%) studies reported the effects of the intervention on all measures of the outcome using graphed data.

Reliability during data collection was calculated for seven (70%) studies. Interobserver agreement was calculated by dividing the number of agreements by the number of agreements plus disagreements, then multiplying by 100. The overall reliability range across studies were 91–100%. Measures used were independent and simultaneous recorded observation (n = 4, 40%) during data collection and written research protocol (n = 2, 20%) regarding data collection procedures. Social validity was reported at the end of seven (70%) studies using questionnaires (n = 3, 30%), surveys (n = 2, 20%), and interviews (n = 2, 20%). Positive responses noted across the seven studies indicated that AT interventions were effective and preferred over other assistance (Collins, Ryan, Katsiyannis, Yell, & Barrett, 2014; Douglas & Uphold, 2014; Green et al., 2011; McMahon, Cihak, & Wright, 2015; McMahon et al., 2015; Van Laarhoven et al., 2007; Van Laarhoven, Johnson, Van Laarhoven-Myers, Grider, & Grider, 2009). Four (40%) studies that did not meet the QI-7 were: (a) Chang et al. (2011), because inter-observer reliability and social validity were not evidenced in the study; (b) Van Laarhoven et al. (2009), because of insufficient baseline data collection for 2 of the 3 tasks; and (c) Cihak et al. (2007) and Cihak et al. (2008), did not report social validity.

Data analysis (QI-8)

Given every study utilized a single-case design, the authors needed to provide graphs of outcome data at the unit of analysis for all phases that allow for visual analysis of experimental control. All 10 (100%) studies met the data analysis criteria. Of the 10 intervention studies reviewed in this analysis, the effectiveness of the intervention was evidenced in all the studies. However, only two (20%) studies reported ES. Collins, Ryan, Katsiyannis, Yell, and Barrett (2014) calculated ES using percentage of nonoverlapping data (PND) between the baseline and successive intervention phases. The analysis of ES resulted in a 100% PND, meaning the AT was highly effective. Green, Hughes, and Ryan (2011) calculated ES using percentage of data point exceeding the median of baseline phase approach for each pair of baseline and treatment phases. ES calculated demonstrated a large (1.0) change between phases suggesting change in the outcome measure was greater. While none of the other studies reported ES, Chang et al. (2011) reported the difference between the baseline and intervention phase was statistically significant (p < .05).

According to the CEC, methodologically sound studies need to meet all the QIs. Unfortunately, there were only two (20%) studies (Collins et al., 2014; Douglas & Uphold, 2014) that met every QI. Overall, research included in all the studies demonstrated successful positive outcomes using AT prompting devices.

Treatment Outcomes

Table 3 provides a detailed overview for the studies that provided sufficient information to calculate visual analysis, PND, SMD, and Tau-U. Overall, the use of AT support for individuals with ID resulted in medium to highly effective outcomes for improving employment skills (PND 91%; SMD 2.84; Tau-U 0.902). The effectiveness of AT was further evaluated based on skills addressed as well as the types of technology used.

Table 3.

Effects of AT on Employment for Individuals With Intellectual Disability.

Study	Type of AT	Visual Analysis	PND (%)	SMD (Variance)	95% CI	Tau-U (Variance)	95% CI
Cihak, Kessler, and Alberto (2007)	Handheld computer	16/16	92	3.0 (0.01)	[2.9, 3.1]	0.92 (0.07)	[0.78, 1]
Van Laarhoven, Van Laarhoven-Myers, and Zurita (2007)	Handheld computer	12/12	100	3.0 (0.01)	[2.9, 3.1]	0.90 (0.08)	[0.74, 1]
Cihak, Kessler, and Alberto (2008)	Handheld computer	4/4	100	3.0 (0.01)	[2.9, 3.1]	0.96 (0.17)	[0.63, 1]
Van Laarhoven, Johnson, Van Laarhoven-Myers, Grider, and Grider (2009)	Handheld computer	1/1^b	100	^a	^a	0.92 (0.33)	[0.26, 1]
Chang, Chen, and Lu (2011)	Portable electronic device	2/2	88	2.5 (1.0)	[1.5, 3.5]	0.97 (0.15)	[0.69, 1.0]
Green, Hughes, and Ryan (2011)	Wearable technology	1/1	50	2.04 (1.69)	[0.2, 3.9]	0.66 (0.36)	[0.96, 1]
Collins, Ryan, Katsiyannis, Yell, and Barrett (2014)	Portable electronic device	9/9	100	3.0 (0.01)	[2.9, 3.1]	1.0 (0.14)	[0.72, 1]
Douglas & Uphold (2014)	Portable electronic device	5/5	100	3.0 (0.01)	[2.9, 3.1]	0.99 (0.18)	[0.64, 1]
McMahon, Cihak, and Wright (2015)	Portable electronic device	4/4	100	3.0 (0.01)	[2.9, 3.1]	1.0 (0.19)	[0.64, 1]
McMahon et al. (2015)	Portable electronic device	6/6	100	3.0 (0.01)	[2.9, 3.1]	0.70 (0.13)	[0.44, 0.95]

Note. PND = percentage of nonoverlapping data (<50% unreliable, 50–70% questionable, 70–90% fairly effective, and >90% highly effective); SMD (0–0.3 is small, 0.3–0.8 is medium, and >0.8 is large) = standardized mean difference; AT = assistive technology; CI = confidence interval.

^aEffect size could not be calculated due to insufficient number of baseline data points.

Skills addressed

When evaluating the effectiveness of AT based on the types of employment tasks addressed, AT showed the largest ES gains for studies (n = 3) focusing on improving work task completion (PND 99%; SMD 3.0; Tau-U 0.95). Work tasks included cleaning bathrooms; rolling silverware; and even office tasks such as scanning, faxing, and photocopying a document. Studies that utilized AT to help an individual with ID navigate independently to a potential employment setting (n = 2), showed slightly smaller medium to large ES gains (PND 100%; SMD 3.0; Tau-U 0.85). Both of these studies reported participants made more independent navigation decisions using the augmented reality navigation tool and reached unknown locations without needing any assistance. Similar positive results were also obtained from a single study investigating the effects of AT on employee productivity (PND 88%; SMD 2.5; Tau-U 0.97) by helping monitor employee break times. The least consistent findings when comparing ES methodologies was observed in a single study that used AT to improve time management skills (PND 50%; SMD 2.04; Tau-U 0.66) due to the high variability in subject performance. However, results indicated that a vibrating watch enabled an individual with ID to leave work and arrive to class on time without requiring assistance or prompting from a job coach.

Types of technology

Lastly, the effectiveness among AT devices also varied. Studies utilizing portable electronic devices (n = 6) showed large or highly effective ES gains (PND 98%; SMD 2.9; Tau-U 0.93). Over half (60%) of the studies reviewed used portable electronic devices ranging from the earliest iPod touch generations to more recent versions of smartphones and iPads. These portable electronic devices were used in conjunction with a number of different types of application ranging from a functional planning system to a G-sensor technology that monitors subject activity levels. Three studies utilized handheld computers which resulted in a large or highly effective ES gains (PND 97%; SMD 3.0; Tau-U 0.93). These studies indicate handheld computers with picture, auditory, and video prompts improved transitions between tasks as well as the ability to independently and correctly complete increasingly complex tasks. Lastly, wearable technologies (n = 1) resulted in the least consistent gains due to high variability of student performance (PND 50%; SMD 2.04; Tau-U 0.66). This study used a vibrating watch to improve the ability of an employee with ID to leave and arrive on time for work.

Discussion

In this review, we examined the research base for utilizing AT support in improving employment skills for individuals with ID. Based on available studies, several notable findings are discussed, including the positive outcomes of AT for employment and a recent shift in technology trends. First, despite the methodology selected for calculating ES, the use of AT for increasing employment outcomes for individuals with ID would be judged beneficial, with pronounced positive changes in performance across all tasks. When outcomes were analyzed by either the type of AT used (i.e., handheld computers, wearable technology, and portable electronic devices) or task completed (i.e., navigation and prompts), meaningful ES gains were achieved. The variety of work tasks to which technologies were applied illustrates the potential that technology can play in supporting employment and vocational skills. Second, there has been a distinct shift in the type of ATs over the past decade moving from handheld computers to portable electronic devices such as iPods, iPads, and applications on smartphones. The shift to more advanced technology has been made apparent through this review. Video prompts evolved from software installed on a handheld computer to iPods and iPads. This trend in using portable electronic devices as AT has blurred the line between mainstream technology and AT, as many students without disabilities are using their smartphones to assist them in a variety of ways (Fichten et al., 2014). Other advantages such as affordability, availability, technical support, and low concerns about social stigma are also leading this trend (Chang & Wang, 2010).

Limitations

The following limitations should be considered when interpreting the results of this systematic review. First, although a comprehensive list of terms associated with AT was used in the search, there may have been some studies identified by another name that were not included. Second, all studies were limited to small sample sizes, with no more than six participants in any study. Even though the studies reported positive effects of technology on an individual’s ability to work independently in an employment setting, it is difficult to generalize from a limited number of studies to the entire population of ID. A third limitation was the length of the studies. Some participants were only involved for a limited amount of time, and the studies did not measure long-term maintenance of skills gained. Maintenance of skills measured ranged from only 2–9 weeks. Findings may be stronger if the studies were extended over a longer period of time and if long-term maintenance was included.

Recommendations for Future Research

The majority of studies focused on task completion, hence future research should focus on utilizing AT to strengthen time management skills and navigational skills for individuals with ID given there were only three studies (Green et al., 2011; McMahon et al., 2015; McMahon et al., 2015) related to these topics. Recent research has highlighted the fact that time management continues to be a significant barrier in acquiring and maintaining employment for individuals with ID (Fichten et al., 2014).

Future research should also focus on the use of emerging technology such as augmented reality and virtual reality to support individuals with ID to build their employment skills. Only two studies (McMahon et al., 2015; McMahon et al., 2015) focused on augmented reality and no studies investigated virtual reality. Augmented reality is a technology that layers computer-generated enhancements atop an existing reality in order to make it more meaningful through the ability to interact with it (Craig, 2013). In contrast, virtual reality is an artificial, computer-generated simulation of a real-life environment or situation. It immerses the user by making them feel like they are experiencing the simulated reality firsthand, by stimulating their vision or hearing (Ji & Zhang, 2016). Virtual reality may prove empowering for individuals with ID, providing them an opportunity to practice and acquire new skills; especially abstract concepts that can be difficult to comprehend (Jeffs, 2015).

It is important to note only two studies reported an ES and nearly half of the studies consistently overlooked three of the eight CEC QIs. There is an increased need to calculate data sets from single-case designs due to the increased attention to evidence-based practice (Olive & Franco, 2008). In order to support the use of single-case research as evidence-based, it is crucial for researchers to synthesize data, as it would be detrimental for teachers to use only recommendations brought forth from group design research (Olive & Franco, 2008). Without the calculation of an ES, researchers conducting systematic reviews may have greater difficulty in interpreting and comparing intervention effectiveness of single-subject studies (Olive & Franco, 2008). Researchers can improve the quality of future research by ensuring they focus on the standards for evidence-based practices in special education (CEC, 2014).

Another consideration for future researchers is the current lack of individualized AT assessment prior to the selection of the AT devices used by the participants. This lack of assessment suggests that some studies may not be effectively matching students’ needs and features of AT equipment. Research indicates that student’s feelings about the AT and the support of family, peers, and teachers are critical factors that can determine successful use versus abandonment (Schwartzberg, Kakavas, & Malkind, 1996). Other factors influencing abandonment include being able to use the AT device with little or no pain, fatigue, discomfort, or stress (Schwartzberg et al., 1996).

An additional focus would involve focusing on implementing AT interventions earlier in school vocational programs to improve employment skills and potentially increase employability. Research that focused on implementing AT interventions in high school settings may be beneficial as these individuals are preparing to transition to the workforce in the near future.

All participants in the 10 studies were diagnosed with mild to moderate IDs, thus suggesting that individuals with significant learning impairments can be taught to use technology to improve independence and overall employment skills. A recommendation from this systematic review would be to extend research to include studies on individuals with more profound and/or multiple disabilities.

Conclusion

One of the biggest challenges facing individuals with ID is the ability to acquire and maintain gainful employment (Siperstein, Heyman, & Stokes, 2014). Employment has been linked to many positive aspects including increased levels of quality of life, independence, and self-esteem (Lehman, Goldberg, Dixon, & McDonnell, 2002). As a whole, this body of research has demonstrated the positive effects of prompting by technology on an individual’s ability to independently respond correctly in an employment setting. This was illustrated across a range of employment settings and various types of technology and prompts. These findings are consistent with those of prior reviews. For example, Cullen, Alber-Morgan, and Shelia (2015) found that prompting by technology was more effective than low-tech methods when teaching tasks. Similarly, Sauer et al. (2010) indicated that following the implementation of AT, there was an increase in accuracy, independence, and generalization of skills in the workplace. Even though portable electronic devices are now more readily accessible than ever before, their potential continues to be underutilized by individuals with ID. Enabling persons with ID to take full advantage of AT is essential for their overall success (at home, school, and work; Cameto, Levine, & Wagner, 2004). Preparation for employment needs to be a primary research focus with the emphasis on how AT can provide the supports needed to facilitate this process.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

American Association on Intellectual and Developmental Disabilities. (2013). Intellectual disability: Definition, classification, and systems of supports (11th ed.). Retrieved from http://aaidd.org/intellectual-disability/

Beyer

Perry

(2013). Promoting independence through the use of assistive technology. Tizard Learning Disability Review, 18, 179–185.

Busk

P. L.

Serlin

R. C.

(2005). Meta-analysis for single-case research. In Kratochwill

T. R.

Levin

J. R.

(Eds.), Single-case research design and analysis: New directions for psychology and education (pp. 187–212). Hillsdale, NJ: Lawrence Erlbaum.

Cameto

Levine

Wagner

(2004). Transition planning for students with disabilities. A special topic report of findings from the National Longitudinal Transition Study-2 (NLTS2). Menlo Park, CA: SRI International.

Cannella-Malone

Sigafoos

O’Reilly

De la Cruz

Edrisinha

Lancioni

G. E.

(2006). Comparing video prompting to video modeling for teaching daily living skills to six adults with developmental disabilities. Education and Training in Developmental Disabilities, 41, 344–356.

*Chang

Y. J.

Chen

S. F.

Z. Z.

(2011). An accelerometer-based handheld system to reduce breaks in performance of young adults with cognitive impairments. Research in Developmental Disabilities, 32, 2530–2534.

Chang

Y.-J.

Chou

L.-D.

Wang

F. T.-Y.

Chen

S.-F.

(2013). A kinect-based vocational task prompting system for individuals with cognitive impairments. Personal and Ubiquitous Computing, 17, 351–358.

Chang

Y. J.

Wang

T. Y.

(2010). Comparing picture and video prompting in autonomous indoor wayfinding for individuals with cognitive impairments. Personal and Ubiquitous Computing, 14, 737–747.

*Cihak

D. F.

Kessler

Alberto

P. A.

(2008). Use of a handheld prompting system to transition independently through vocational tasks for students with moderate and severe intellectual disabilities. Education and Training in Developmental Disabilities, 43, 102–110.

10.

*Cihak

D. F.

Kessler

K. B.

Alberto

P. A.

(2007). Generalized use of a handheld prompting system. Research in Developmental Disabilities, 28, 397–408.

11.

Citron

Brooks-Lane

Crandell

Brady

Cooper

Revell

(2008). A revolution in the employment process of individuals with disabilities: Customized employment as the catalyst for system change. Journal of Vocational Rehabilitation, 28, 169–179.

12.

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum.

13.

*Collins

J. C.

Ryan

J. B.

Katsiyannis

Yell

Barrett

D. E.

(2014). Use of portable electronic assistive technology to improve independent job performance of young adults with intellectual disability. Journal of Special Education Technology, 29, 15–29.

14.

Cook

B. G.

Buysse

Klinger

Landrum

T. J.

McWilliams

R. A.

Tankersley

Test

D. W.

(2015). CEC’s standards for classifying the evidence base of practices in special education. Remedial and Special Education, 36, 220–234.

15.

Council for Exceptional Children. (2014). Council for Exceptional Children: Standards or evidence-based practices in special education. Retrieved from http://www.cec.sped.org/∼/media/Images/Standards/CEC%20EBP%20Standards%20cover/CECs%20Evidence%20Based%20Practice%20Standards.pdf

16.

Craig

A. B.

(2013). Understanding augmented reality: Concepts and applications. Amsterdam, the Netherlands: Morgan Kaufmann.

17.

Cullen

Alber-Morgan

Shelia

(2015). Technology mediated self-prompting of daily living skills for adolescents and adults with disabilities: A review of the literature. Education and Training in Autism and Developmental Disabilities, 50, 34–55.

18.

Davies

D. K.

Stock

S. E.

Wehmeyer

M. L.

(2002). Enhancing independent time-management skills of individuals with mental retardation using a palmtop personal computer. Mental Retardation, 40, 358–365.

19.

Davies

D. K.

Stock

S. E.

Wehmeyer

M. L.

(2003). A palmtop computer-based intelligent aid for individuals with intellectual disabilities to increase independent decision making. Research & Practice for Persons with Severe Disabilities, 28, 182–193.

20.

*Douglas

K. H.

Uphold

N. M.

(2014). IPad or iPod touch: Evaluating self-created electronic photographic activity schedules and student preferences. Journal of Special Education Technology, 29, 1–14.

21.

Ferguson

Myles-Smith

Hagiwara

(2005). Using a personal digital assistant to enhance the independence of an adolescent with asperger syndrome. Education and Training in Developmental Disabilities, 40, 60–67.

22.

Fichten

C. S.

Asuncion

Scapin

(2014). Digital technology, learning, and postsecondary students with disabilities: Where we’ve been and where we’re going. Journal of Postsecondary Education and Disability, 27, 369–379.

23.

Furniss

Lancioni

Rocha

Cunha

Seedhouse

Morato

O’Reilly

M. F.

(2001). VICAID: Development and evaluation of palmtop-based job aid for workers with severe developmental disabilities. British Journal of Educational Technology, 32, 277–287.

24.

Gast

D. L.

(2005). Visual analysis of graphic data. In Sugai

Horner

(Eds.), Encyclopedia of behavior modification and cognitive behavior therapy: Educational applications (Vol. 3, pp. 1595–1599). Thousand Oaks, CA: Sage.

25.

Gilson

C. B.

Carter

E. W.

Biggs

E. E.

(2017). Systematic review of instructional methods to teach employment skills to secondary students with intellectual and developmental disabilities. Research and Practice for Persons with Severe Disabilities, 42, 89–107.

26.

Gingerich

W. J.

(1984). Methodological observations on applied behavioral science. The Journal of Applied Behavioral Science, 20, 71–79.

27.

*Green

J. M.

Hughes

E. M.

Ryan

J. B.

(2011). The use of assistive technology to improve time management skills of a young adult with an intellectual disability. Journal of Special Education Technology, 26, 13–20.

28.

Individuals with Disabilities Education Act, 20 U.S.C. § 1400 (2004).

29.

Jeffs

T. L.

(2015). Virtual reality and special needs. Themes in Science and Technology Education, 2, 253–268.

30.

Zhang

(2016). Integration and application of animation production simplification in a virtual reality teaching system. Internal Journal of Emerging Technologies in Learning, 11, 37–41.

31.

Kavale

K. A.

Mathur

S. R.

Forness

S. R.

Quinn

M. M.

Rutherford

R. B.

(2000). Right reason in the integration of group and single-subject research in behavioral disorders. Behavioral Disorders, 25, 142–157.

32.

Kratochwill

T. R.

Hitchcock

J. H.

Horner

R. H.

Levin

J. R.

Odom

S. L.

Rindskopf

D. M

Shadish

W. R.

(2010). Single case designs technical documentation. In What Works Clearinghouse: Procedures and standards handbook (version 2.0). Retrieved from http://ies.ed.gov/ncee/wwc/pdf/wwc_procedures_v2_standards_handbook.pdf

33.

Lehman

A. F.

Goldberg

Dixon

McDonnell

(2002). Improving employment outcomes for persons with severe mental illness. Archives of General Psychiatry, 59, 165–172.

34.

Liberati

Altman

D. G.

Tetzlaff

Mulrow

Gøtzsche

P. C.

Ioannidis

J. P.

… Moher

(2009). The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate health care interventions: Explanation and elaboration. PLoS Medicine, 6, e1000100. doi:10.1371/journal.pmed.1000100

35.

*McMahon

Cihak

D. F.

Wright

(2015). Augmented reality as a navigation tool to employment opportunities for post-secondary education students with intellectual disabilities and autism. Journal of Research on Technology in Education, 47, 157–172.

36.

*McMahon

D. D.

Smith

C. C.

Cihak

D. F.

Wright

Gibbons

M. M.

(2015). Effects of digital navigation aids on adults with intellectual disabilities: Comparison of paper map, Google maps, and augmented reality. Journal of Special Education Technology, 30, 157–165.

37.

Mechling

(2011). Review of twenty-first century portable electronic devices for persons with moderate intellectual disabilities and autism spectrum disorders. Education and Training in Autism and Developmental Disabilities, 46, 479–498.

38.

Newman

Wagner

Knokey

A.-M.

Marder

Nagle

Shaver

Schwarting

(2011). The post-high school of young adults with disabilities up to 8 years after high school. A report from the National Longitudinal Transition Study-2 (NLTS2) (NCSER 2011-3005). Menlo Park, CA: SRI International.

39.

Nota

Ginevra

M. C.

Carrieri

(2010). Career interests and self-efficacy beliefs among young adults with an intellectual disability. Journal of Policy and Practice in Intellectual Disabilities, 7, 250–260.

40.

Olive

M. L.

Franco

J. H.

(2008). Effect size matters: And so does the calculation. The Behavior Analyst Today, 9, 5–10.

41.

Parker

R. I.

Vannest

K. J.

(2012). Bottom up analysis of single-case research designs. Journal of Behavioral Education, 17, 254–265.

42.

Parker

R. I.

Vannest

K. J.

Brown

(2009). The “improvement rate difference” for single-case re-search. Exceptional Children, 75, 135–150.

43.

Parker

R. I.

Vannest

K. J.

Davis

J. L

. (2011). Effect size in single-case research: A review of nine nonoverlap techniques. Behavior Modification, 35, 303–322.

44.

Parker

R. I.

Vannest

K. J.

Davis

J. L.

Sauber

S. B.

(2011). Combining non-overlap and trend for single case research: Tau-U. Behavior Therapy, 42, 284–299.

45.

Perone

(1993). Statistical inference in behavior analysis. Experimental control is better. The Behavior Analyst, 22, 109–116.

46.

Rakap

(2015). Effect sizes as result interpretation aids in single-subject experimental research: Description and application of four nonoverlap methods. British Journal of Special Education, 42, 11–33.

47.

Riffel

Wehmeyer

M. L.

Turnball

A. P.

Lattimore

Davies

Stock

Fisher

(2005). Promoting independent performance of transition related tasks using a palmtop pc-based self-directed visual and auditory prompting system. Journal of Special Education Technology, 20, 5–14.

48.

Rose

Saunders

Hensel

Kroese

B. S.

(2005). Factors affecting the likelihood that people with intellectual disabilities will gain employment. Journal of Intellectual Disabilities, 9, 9–23.

49.

Sauer

A. L.

Parks

Heyn

P. C.

(2010). Assistive technology effects on the employment outcomes for people with cognitive disabilities: A systematic review. Disability and Rehabilitation: Assistive Technology, 5, 377–391.

50.

Schalock

R. L.

Brown

Cummins

R. A.

Felce

Matikka

Parmenter

(2002). Conceptualization, measurement, and application of quality of life for persons with intellectual disabilities: Report of an international panel of experts. Mental Retardation, 40, 457–470.

51.

Schwartzberg

J. G.

Kakavas

V. L.

Malkind

(1996). Guidelines for the use of assistive technology: Evaluation, referral, prescription (2nd ed.). Chicago, IL: American Medical Association.

52.

Scruggs

T. E.

Mastropieri

M. A.

(2013). PND at 25: Past, present, and future trends in summarizing single subject research. Remedial and Special Education, 34, 9–19. doi:10.1177/074193251244-730

53.

Shadish

W. R.

Hedges

L. V.

Horner

R. H.

Odom

S. L.

(2015). The role of between-case effect size in conducting, interpreting, and summarizing single-case research (NCER 2015-002). Washington, DC: U.S. Department of Education, National Center for Education Research, Institute of Education Sciences (IES).

54.

Siperstein

G. N.

Heyman

Stokes

J. E.

(2014). Pathways to employment: A national survey of adults with intellectual disabilities. Journal of Vocational Rehabilitation, 41, 165–178.

55.

Smith

T. L.

Polloway

E. A.

Smith

J. D.

Patton

R. E.

(2007). Self-determination for persons with developmental disabilities: Ethical considerations for teachers. Education & Training in Developmental Disabilities, 42, 144–151.

56.

Soares

D. A.

Harrison

J. R.

Vannest

K. J.

(2016). Effect size for token economy use in contemporary classroom settings: A meta-analysis of single case research. School Psychology Review, 45, 379–399.

57.

*Van Laarhoven

Johnson

J. W.

Van Laarhoven-Myers

Grider

K. L.

Grider

K. M.

(2009). The effectiveness of using a video Ipod as a prompting device in employment settings. Journal of Behavior Education, 18, 119–141.

58.

Van Laarhoven

Van Laarhoven-Myers

(2006). A comparison of three video-based instructional procedures for teaching daily living skills to persons with developmental disabilities. Education and Training in Developmental Disabilities, 41, 365–381.

59.

*Van Laarhoven

Van Laarhoven-Myers

Zurita

L. M.

(2007). The effectiveness of using a pocket PC as a video modeling and feedback device for individuals with developmental disabilities in vocational settings. Assistive Technology Outcomes and Benefits, 4, 28–45.

60.

Vannest

K. J.

Parker

R. I.

Gonen

. (2011). Single case research: Web based calculators for SCR analysis (Version 1.0) [Web-based application]. College Station: Texas A&M University. Retrieved from singlecaseresearch.org

61.

Wehman

P. H.

(2011). Employment for persons with disabilities: Where are we now and where do we need to go? Journal of Vocational Rehabilitation, 35, 145–151.

62.

Wehmeyer

M. L.

Palmer

S. B.

Smith

S. J.

Parent

Davies

D. K.

Stock

(2006). Technology used by people with intellectual and developmental disabilities to support employment activities: A single subject design meta analysis. Journal of Vocational Rehabilitation, 24, 81–86.

63.

Wolery

Busick

Reichow

Barton

(2010). Comparison of overlap methods for quantitatively synthesizing single-subject data. Journal of Special Education, 44, 18–28.