Impact and Policy Implications of Transition Education Practices on IDEA Indicator 13 Scores and Indicator 14 Outcomes

Abstract

U.S. federal special education policies require state departments of education to collect data to measure progress in attaining state performance plan goals, with Indicators 13 and 14 measuring the Individualized Education Program (IEP) transition process and post-high school student outcomes. Research suggests that higher transition indicator scores result in better post-school outcomes, but no research has yet demonstrated the impact transition education practices have upon indicator scores. This study identified transition education practices that impacted one state’s transition Indicators 13 (transition IEP process compliance scores) and 14 (post-high school outcomes). Eighty-three secondary special educators from 36 districts identified the extent their students engaged in various evidence-based transition practices. Hierarchical cluster analysis identified five transition education themes. Multivariate analysis of variance revealed few statistically significant results and several meaningful effect sizes indicating specific transition education practices that impacted Indicator 14 outcomes. Districts with students actively involved in IEP meetings had higher Indicator 13 scores and 14 outcomes. Districts with high to moderate number of students with paid jobs and money management experience had higher Indicator 14 outcomes. Districts that taught students to set postsecondary goals had higher Indicator 14 outcomes. Other results are described and policy implications discussed.

Keywords

transition transition practices Indicator 13 Indicator 14

The purpose of the United States’ Individuals with Disabilities Education Improvement Act of 2004 (IDEA) includes preparation of students with disabilities for employment, postsecondary education, and independent living. To attain this outcome, educators and other Individualized Education Program (IEP) plan members need to consider’ interests, strengths, and needs when deciding on transition goals and services. To assist, IDEA 2004 mandates using transition assessments and wants students to become engaged in the transition planning process. The cumulative effect of this and other federal mandates established transition education as a focus of secondary special education, laid the foundation for increased student involvement in transition planning, and created the need for comprehensive transition education practices to improve students’ post-school outcomes (e.g., Kohler et al., 2016; Rusch et al., 2009).

Districts across the country use Kohler et al.’s (2016) Taxonomy for Transition Programming 2.0: A Model for Planning, Organizing, and Evaluating Transition Education, Services, and Programs to plan transition practices and improve student post-school outcomes. The taxonomy consists of five components: (a) student development, (b) family involvement, (c) program structure, (d) interagency collaboration, and (e) student-focused planning. This framework of essential transition education components provides students with disabilities needed skills, experiences, and programs to increase their likelihood for positive postsecondary outcomes.

Many of the evidence- and research-based practices included in Kohler et al.’s (2016) taxonomy have been shown to predict post-school success (e.g., Mazzotti et al., 2021; Rowe et al., 2021). To give some examples, Wei et al. (2016) found students who attained transition goals achieved better graduation and employment rates. Self-determination skills that enable individuals to make choices, solve problems, set goals and evaluate their progress, and self-advocate (Rowe et al., 2015) are part of a growing body of research indicating a strong connection between those skills and stable post-high school employment and further education (Shogren et al., 2015). Students who actively participate in their IEP and transition planning using evidence-based transition practices (e.g., Self-Directed IEP; Martin et al., 2016) have higher self-determination skill levels than those who do not engage in IEP transition planning activities (Seong et al., 2015). Finally, students with a history of work experiences (Papay & Bambara, 2014), unpaid and paid work experiences (Joshi et al., 2012), and paid work history show a greater likelihood of post-school employment (Carter et al., 2012).

Transition Indicators 13 and 14

After IDEA’s reauthorization in 2004, the U.S. Department of Education’s Office of Special Education required states to establish performance targets, two of which focused on transition practices and post-high school outcomes (Erickson et al., 2014). Indicator 13 measures compliance with IEP transition components, such as postsecondary transition goals based on age-appropriate transition assessments and student attendance at IEP meetings when transition issues are discussed. Indicator 14 measures the percent of youth who had IEPs employed or enrolled in postsecondary education after high school. States vary in their collection methods (Gerber et al., 2014).

Erickson et al. (2014) examined the relationship between Indicator 13 scores and Indicator 14 further education and employment outcomes and found former students with IEPs who graduated with higher Indicator 13 scores were more likely to have completed a semester of college or a vocational training program. The researchers found no statistically significant impact of Indicator 13 results on post-high school employment outcomes. These findings suggest a strong need for additional research to examine the impact of quality transition planning on post-school outcomes.

Our purpose was to examine the impact of select evidence-based transition education practices upon transition Indicator 13 scores and Indicator 14 outcomes and answer the following questions: What categories of transition practices can be observed? and Do districts with higher implementation of categories of transition education best practices have higher Indicator 13 scores and better Indicator 14 post-school outcomes? To do this, we surveyed district personnel to determine whether or not including more transition education components across a district would be associated with higher Indicator 13 scores and/or higher Indicator 14 outcomes.

Method

Participants and Setting

This study involved a state in the southwestern part of the United States that agreed to provide de-identified Indicators 13 and 14 data for all public school districts. We invited secondary special educators and other transition personnel from all 89 public school districts within the state to participate. We obtained special educator contact information from 66 (74%) of the districts and sent email participation invitations to those transition educators once per week for 3 weeks. A total of 83 special educators (i.e., 22 transition specialists, 48 special education teachers, 13 special education coordinators) directly involved with secondary transition education from 36 rural and urban districts from across the state participated in this study, which represents a 40% district-level response rate. The enrollment of responding school districts varied between approximately 96,000 and 110 students ( $\bar{x}$ = 7,019, s = 16,043). Whereas 20 teachers voluntarily responded from the largest school district, most other districts had only one (n = 23) or two (n = 6) participating educators. Student demographic characteristics varied across districts, with enrollment ranges for White (4.80%–73.50%), Hispanic (2.20%–95.00%), and Native American (0.00%–89.20%) varying across school districts. Due to state limitations, we were unable to collect additional educator demographic information.

Development of the Transition Education Implementation Assessment Survey

We first reviewed the literature to identify evidence-based transition education practices (e.g., Kohler et al., 2016; Mazzotti et al., 2021; Test et al., 2009) around which survey items were developed. The survey had four main sections: (a) transition planning (13 items), (b) vocational education (five items), (c) interagency collaboration (one item), and (d) student and family involvement (five items). Items in these sections were developed because they had substantial research support (e.g., Mazzotti et al., 2021; National Technical Assistance Center on Transition [NTACT], 2016) and were the most prevalent practices in the state where data was collected (as observed by the study’s third author). Eighteen practicing transition educators not involved in the study reviewed draft survey items for clarity and wording, with identified changes made. Next, two colleagues familiar with both survey construction and transition research/practice provided suggested item improvements. Revised items were checked against original studies to ensure alignment. For all items, transition professionals responded to, “___ number of students out of a total of ___,” to indicate the number of students on their caseloads who received or engaged in the stated transition education practice, from which percentages were calculated. Cronbach’s alpha for the survey’s 24 items was .84, suggesting reasonable internal consistency. Item-total correlations ranged between .82 and .86.

Procedure

To maintain confidentiality, a research support office hosted the survey, sent emails to the identified secondary educators, reviewed data to ensure deidentification prior to data analysis, and used computer IP addresses to ensure that duplicate email requests were not sent to participants who already responded. The survey web site automatically recorded respondents’ answers into a spreadsheet.

We used district-level Indicators 13 and 14 scores from the statewide State Performance Plan collected the year prior to this study. The state contracted independent evaluators, mostly retired special educators who completed a training program, to complete IEP reviews. Evaluators individually examined students’ files to indicate if IEPs contained the required Indicator 13 information using NTACT’s Indicator 13 Checklist: Form B (NTACT, n.d.). The number of “yes” marked items were divided by the total number of transition items in the IEP and then multiplied by 100 to obtain the percent score for each IEP. The evaluators then averaged each IEP transition section percentage to determine each district’s Indicator 13 score.

We used the state’s Indicator 14 data gathered 1 year after students graduated or exited high school, which was collected by students’ former high school special educators through paper surveys, phone conversations, and face-to-face interviews. The Indicator 14 data provided to the researchers had only two summary variables: (a) the percentage of students from the districts employed after graduation and (b) the percentage of students enrolled in some type of higher education after leaving high school. We averaged these two percentages to create the Indicator 14 percent score for each school district, which decreased the effect of a duplicated count because many former students enrolled in higher education were also employed. Almost all students lived at home so independent living data was not included.

We combined multiple educator responses on the transition practices survey from the same district into an average district response for each item so each district carried the same weight in subsequent analyses. To cluster items from this survey into broader categories of practices used as independent variables for this study, we used hierarchical cluster analysis using Ward’s (1963) method and squared Euclidian distances because the sample size was too low to use factor analytic techniques. This method clustered individual or groups of items having the smallest squared Euclidian distances together and gave results in an agglomeration schedule and visually as a dendrogram. Because Ward’s hierarchical cluster analysis will extend until all items cluster together, we terminated the analysis when items clustered into categories of evidence-based transition practices (e.g., Haber et al., 2016; Kohler et al., 2016; Madaus et al., 2021; Mazzotti et al., 2021; NTACT, 2016) and exhibited moderate correlations. This led to five transition practice clusters (see Table 1): (a) employment preparation, (b) students selecting transition goals, (c) students’ involvement at the IEP meeting, (d) employment and money management experience, and (e) additional transition education practices. Six survey items did not correlate or cluster with others nor make logical or theoretical sense together and were dropped from analyses as independent variables, though a brief description is provided below.

Table 1.

Descriptive Statistics for Final Survey Questions.

Item	$\bar{x}$ (s)
Employment preparation
. . . received formal career exploration instruction?	62.7 (31.8)
. . . had school sponsored career exploration opportunities?	63.9 (32.1)
. . . received formal instruction in how to find employment?	51.4 (31.8)
. . . had school-sponsored opportunities to find employment?	39.4 (32.3)
Transition goal setting
. . . received formal instruction in how to select their annual transition goals?	43.8 (35.7)
. . . selected their own annual transition goals?	43.9 (32.9)
. . . received formal instruction in how to select their postsecondary transition goals?	46.9 (37.6)
. . . selected their own postsecondary transition goals?	55.0 (37.3)
Student involvement in IEP meetings
Of the IEP meetings that you attended in the last year, how many had the student present?	89.7 (17.8)
. . . have completed IEP transition sections?	80.4 (25.0)
When the student attended the meeting, how many expressed their opinion about their goals during transition meeting discussions?	65.0 (27.5)
How many of your students with IEPs received formal instruction on how to express their opinions in their IEP meetings?	49.2 (35.1)
Employment and money management
. . . received formal instruction in how to manage their own money (e.g., budgeting, balancing checkbooks, and comparing prices)?	50.8 (33.3)
. . . managed their own money (e.g., budgeting, balancing checkbooks, and comparing prices)?	46.1 (23.3)
. . . had paid employment?	39.3 (26.6)
Additional transition practices and skills
In the last year, how many of the IEP meetings in which you participated discussed results from recently completed career interest assessments?	55.5 (32.8)
. . . received formal instruction in how to ask teachers for academic help?	61.6 (34.0)
. . . asked their teachers for academic help?	61.2 (27.9)
. . . received formal instruction in daily living skills (e.g., cooking, washing clothes, buy their own groceries)?	52.6 (36.6)
. . . performed daily living skills (e.g., cooking washing clothes, buy their own groceries)?	63.8 (31.7)

Note. Items beginning with “ . . . ” start with the wording, “In the last year, how many of your students with IEPs.” Descriptive statistics are given as percentages. IEP = Individualized Education Program.

Next, we determined high, medium, and low levels of transition education practice for each independent variable by adding and subtracting half a standard deviation to the mean score of that category separately for each variable to form the medium practice category. Districts reporting transition practices above that level were in the high category and those below that level were in the low category for each independent variable. Because these groupings were done by independent variable, numbers of school districts grouped into these three categories will be presented with the results of each variable.

To assess the extent to which districts with high moderate or low levels of transition practices evidenced different Indicator 13 scores and Indicator 14 outcomes, we conducted a multivariate analysis of variance (MANOVA) using Indicators 13 and 14 scores as dependent variables and transition practice item cluster categories as independent variables. Follow-up univariate and Tukey’s post hoc tests (with pairwise comparisons) determined the exact nature of the differences in these groups by the two dependent variables. Because the statistical tests are hampered with low power due to smaller sample sizes, effect sizes were analyzed using Cohen’s (1988) recommendations (i.e., .20 for small effects, .50 for medium effects, .80 for large effects), but these recommendations are understood as being dependent on study context (see Vacha-Haase et al., 2000).

Results

District Transition Indicator Compliance

The percent of Transition Indicator 13 average compliance scores for each district ranged from 0% to 100%, where 0% indicates a rating of no compliance for that district. Combined Indicator 14 outcomes ranged from 12.25% to 72.25% of the former students employed and/or enrolled in further education.

Transition Education Practice Categories

Employment Preparation

Four survey questions provided information about employment preparation practices. District means indicated 63% of the students received career exploration instruction, and an equal number received school-sponsored career exploration opportunities. Respondents from approximately 52% of school districts reported their districts provided instruction in finding employment, which was higher than the 39% of districts reporting provided school-sponsored opportunities to find employment. Bivariate item-level correlations range from .350 to .698 and all are significant at the p < .05 level.

Students Being Taught and Then Selecting Transition Goals

Four items gauged the extent to which district transition programs taught students to select annual and postsecondary transition goals, and if students selected their annual and postsecondary transition goals. The results suggested slight district differences between instruction for choosing annual goals ( $\bar{x}$ = 43.81%, s = 35.69%) and instruction for choosing postsecondary goals ( $\bar{x}$ = 46.89%, s = 37.56%). Educators reported approximately 45% of students selected their annual transition goals ( $\bar{x}$ = 43.92%, s = 32.86%), and 55% of students selected their postsecondary transition goals ( $\bar{x}$ = 54.96%, s = 37.33%). The correlation between instruction and transition goal selection was very strong for annual (r = .931, p < .001) and strong for postsecondary goals (r = .672, p < .001).

Students’ Involvement at IEP Meetings

Across the 36 school districts, 80% reported students completing IEP transition sections. Similarly, about 90% of students were present for their IEP meetings, and 65% discussed their goals during transition meeting discussions. Approximately half of the school districts taught students to express their opinion at their IEP meetings ( $\bar{x}$ = 49.23%, s = 35.14%). The correlation between instruction and action was significant (r = .397, p = .018).

Employment and Money Management Experience

Instruction in money management did not correlate with the act of managing money (r = −.038, p = .840) or having been employed (r = .086, p = .639). Thus, we removed instruction in money management from any further analyses. A significant correlation existed, however, between students having paid employment and managing their own money (r = .373, p = .042). We found that approximately 40% of students represented by these districts had been employed and 46% of the students managed their own money.

Additional Transition Education Practices and Skills

Across the four categories of survey questions, a diverse set of items matching the areas of Kohler et al.’s (2016) Taxonomy related to student development and student-focused planning clustered together. Educators reported that in 55% of IEP meetings participants discussed the results of career interest assessments. More than 60% of students received instruction on how to ask teachers for academic help, and this same number asked for help. Over half the students received instruction in daily living skills, with more than 60% performing daily living skills. We found a significant correlation between these instructional items and action on the skills taught (r = .513, p = .003).

Non-Correlated Items

Five survey items stood alone and did not correlate with other survey items or with each other. Educators reported that participants at 40% of the IEP meetings discussed adaptive behavior assessments, and 51% of the students received money management instruction. Students tended to increasingly use agency services (almost 50%) when representatives from those agencies attended IEP meetings. Almost 90% of IEP meetings had at least one parent or guardian in attendance, and almost half the educators reported parents assisted students in working toward their postsecondary goals. However, parent participation in IEP meetings was not related to parental assistance with attaining postsecondary goals (r = .068, p = .693). Although these items represent important transition skills used in schools, data from school district personnel show they did not view the skills as related to each other or to other skills, likely because the content of the items and skill instruction is quite different across this category. Because no clear category of transition skills could be ascertained, no further analysis of these items was attempted.

Relationship of Transition Education Practices to Indicator 13 and Indicator 14 Outcomes

As stated previously, effect size indicators were interpreted and privileged over results of standard significance tests due to the small sample size in this study as a result of a small population of interest. It is well understood that effect sizes are calculated independently of power due to sample size (e.g., Sullivan & Feinn, 2012; Vacha-Haase et al., 2000). Thus, interpreting effect size statistics is important because they provide information regarding the magnitude and importance of the results and help standardize interpretations across multiple, differently designed studies, particularly those with less statistical power (e.g., Schmidt & Hunter, 1997; Wilkinson & The APA Task Force on Statistical Inference, 1999). Confidence intervals (CIs) surrounding effect sizes were calculated (Lakens, 2014) to supplement and better understand the meaningfulness of results. For all analyses, composite scores were created independently within category. Rather than trying to imperfectly impute missing data, we chose to eliminate districts listwise within each item category if data was missing on an item in that category. Thus, numbers of districts used for each category differs from the total participating due to missing data at the item level.

Relationship of Employment Preparation to Indicators 13 and 14 Scores

We assessed the extent school districts were classified as reporting high (n = 6 districts), moderate (n = 8), or low levels (n = 6) of employment preparation related to Indicators 13 and 14 scores. Multivariate results showed a significant effect and a moderate effect size, Wilks’ Λ = .311, F(4, 18) = 3.566, p = .026, partial η² = .442, power = .762. Univariate follow-up tests revealed no significant differences between school districts with high, moderate, or low levels of employment preparation and Indicator 13 scores, F(2, 10) = 0.973, p = .411, partial η² = .163, power = .174, and the effect size was smaller than desirable (Cohen, 1988). In contrast, univariate follow-up tests assessing differences on Indicator 14 scores revealed significant differences and a moderately large effect size (Cohen, 1988) between school districts by different levels of employment preparation, F(2, 10) = 9.678, p = .005, partial η² = .659, power = .928. Tukey post hoc tests revealed that school districts in the low employment preparation group had higher Indicator 14 scores ( $\bar{x}$ = 45.04%) than did the high ( $\bar{x}$ = 41.49%) and moderate ( $\bar{x}$ = 27.91%) rated districts. Fewer students from schools classified as having a moderate level of employment preparation were employed than from schools in the high and low groups.

Students Being Taught and Selecting Transition Goals

As with employment preparation, six school districts were classified as providing high levels of instruction for students on selecting transition goals, eight were classified as providing medium levels of instruction, and the remaining six were classified as providing low levels of instruction. Multivariate analyses revealed no significant effects of students being taught and selecting transition goals on either of the Indicator scores, Wilks’ Λ = .811, F(4, 18) = 0.496, p = .739, partial η² = .099, power = .141. As evidenced by CIs surrounding effect size calculations, follow-up univariate tests reveal no meaningful effect on Indicator 13 scores based on the extent to which students selected transition goals, F(2, 10) = 0.081, p = .923, partial η² = .016, 90% CI = [.00, .140], power = .059. Although the effect size for Indicator 14 outcomes appears to be small, F(2, 10) = 1.091, p = .373, partial η² = .179, power = .190, the CI for this effect included zero (90% CI = [.00, .40]). Results show no meaningful impact of differing levels of student input into the selection of transition goals on Indicator 14 outcomes.

Students’ Involvement at the IEP Meeting

We classified five school districts as providing high levels of student involvement in the IEP meeting, with nine classified as medium and six as providing low levels of involvement. Multivariate analyses found non-significant differences between categories of districts regarding the level of students’ involvement at the IEP meeting, Wilks’ Λ = .453, F(4, 18) = 2.185, p = .112, partial η² = .327, power = .525. Since power was lower than optimal, coupled with a non-negligible effect size, we conducted univariate follow-up analyses. Univariate follow-up tests revealed no significant differences on Indicator 13 scores between each of the three groups and the level of student involvement, F(2, 10) = 0.436, p = .658, partial η² = .080, power = .103, which included zero in the effect size CI. Univariate follow-up tests on Indicator 14 scores revealed significant differences with a moderate effect size between districts classified as reporting different levels of student involvement in the IEP Meeting, F(2, 10) = 5.202, p = .028, partial η² = .510, 90% CI = [.04, .66], power = .695. The Tukey’s post hoc tests suggest districts classified as having a high level of students actively participating in their IEP meetings ( $\bar{x}$ = 49.65%) had higher Indicator 14 scores than those reporting either a moderate ( $\bar{x}$ = 36.04%) or low ( $\bar{x}$ = 37.03%) level of student IEP meeting participation, showing that districts having a high level of students participating during their IEP meetings tended to have more students employed and/or enrolled in education after high school graduation.

Employment and Money Management Experience

We examined the multivariate differences between three reported levels of employment and money management (low = 5, medium = 13, high = 8) across two Indicator scores. Results showed non-significant differences on either Indicator score, Wilks’ Λ = .484, F(4, 18) = 1.967, p = .143, partial η² = .304, power = .477. Univariate follow-up tests were conducted due to the low sample size. As with other analyses, classifications of schools based on reported levels of employment and money management experience did not produce meaningful Indicator 13 scores, F(2, 10) = 0.516, p = .612, partial η² = .094, 90% CI = [.00, .30], power = .113. Univariate follow-up tests revealed statistically significant and meaningful differences between each of the three levels of school district student participation and Indicator 14 scores, F(2, 10) = 4.35, p = .044, partial η² = .465, 90% CI = [.09, .63], power = .612. Tukey’s post hoc tests revealed that districts reporting a low level of students with employment and money management experience ( $\bar{x}$ = 30.19%) had lower post-school outcome Indicator 14 scores. Districts classified as having high ( $\bar{x}$ = 43.64%) and moderate ( $\bar{x}$ = 47.45%) levels of employment and money management experience were only different from each other to a small extent, but both had higher Indicator 14 employment and further education outcome scores than the districts where students had low levels of employment or money management experiences.

Additional Transition Skills

Finally, the multivariate analysis shows that school districts who reported different levels of offering additional transition skills instruction (n = 5 high school districts, n = 7 medium districts, n = 8 low districts) did not experience significant differences on Indicator scores, Wilks’ Λ = .688, F(4, 18) = 0.927, p = .470, partial η² = .171, power = .236. We again felt that the non-negligible effect size was high enough to warrant further data analysis and investigation. However, univariate follow-up tests revealed that school district classification for neither Indicator 13, F(2, 10) = 0.040, p = .961, partial η² = .008, 90% CI = [.00, .10], power = .055, nor Indicator 14, F(2, 10) = 2.217, p = .160, partial η² = .307, 90% CI = [.00, .51], power = .349, had a large enough effect size to be considered meaningful, showing no meaningful differences between schools offering different levels of additional transition skill instruction.

Discussion

We undertook this study to determine possible connections between implementation of differing levels of transition education components across school districts, Indicator 13 IEP transition process and documents, and Indicator 14 post-school engagement outcomes. The pattern of meaningful effect sizes from 40% of districts in one state indicates a strong association between transition practices and Indicator 14 post-school education and employment outcomes.

Relations Between Transition Practices, Indicator 14 Outcomes, and Indicator 13 Scores

In relation to Indicator 14 post-school employment and further education outcomes: (a) districts with high levels of student involvement in IEP meetings had more former students employed and/or enrolled in higher education, (b) districts with high to moderate numbers of students with paid employment histories during school years and money management experience had more former students employed and/or engaged in higher education, and (c) districts with the lowest reported numbers of students involved in employment preparation had the lowest number of former students with IEPs employed or enrolled in higher education. The inclusion of zero in the CIs for all Indicator 13 analyses suggests no meaningful results when investigating the relations between transition practices and Indicator 13 scores in this sample.

Relationship of Student Involvement in IEP Transition Meetings on Indicator Outcomes

Transition Indicator 14

It is likely that increased levels of student self-determination contributed to better student post-school outcomes when districts report higher levels of student participation in IEP meetings. Teaching self-determination skills and providing skill practice opportunities for students have become transition education best practices and predictors because of the positive influence on academic performance in public schools (Konrad et al., 2007; Lee et al., 2008), higher education (Ju et al., 2017), and employment (Shogren et al., 2015). Results of this study show that students enrolled in districts reporting higher levels of student participation during their IEP meetings exhibited higher rates of employment and postsecondary enrollment, possibly giving a venue for students to practice the learned self-determination behaviors (e.g., Martin et al., 2006; Seong et al., 2015).

Transition Indicator 13

Although not a statistically significant effect most likely due to low power, higher levels of student IEP involvement did have a moderate positive effect on districts’ Indicator 13 scores. We speculate the increase in reported student involvement led to increased in-depth transition planning discussions and made transition-planning issues more relevant and important. Existing studies support this possibility. Martin et al. (2006) found a statistically significant increase in team members’ knowledge of the transition IEP process when students became more involved in their IEP meetings. Conversely, Powers et al. (2005) reported poorer postsecondary goals with lower levels of student involvement in IEP meetings.

Relations Between Paid Employment and Indicator 14 Outcomes

As with previous work (e.g., McConnell et al., 2012), this study found having paid jobs during high school associated with employment after leaving high school. Having a paid job during the high school years increased workplace socialization skills (Enayati & Karpur, 2019; Wehman et al., 2015) and other work-related competencies, such as self-determination and job skills (Carter et al., 2012) assisted students with postsecondary employment.

Districts with high and moderate levels of student employment preparation opportunities achieved lower Indicator 14 post-school outcomes than districts where fewer students participated in these activities without obtaining a paid job. This questions the efficacy of school-based employment programs that do not lead to paid employment during the high school years.

Impact of Students Being Taught and Then Selecting Transition Goals

We initially believed teaching students to select transition goals would be related to student involvement in the IEP meeting, but it stood alone. With almost half the students being taught and then selecting their transition goals, this transition education practice appears to make a positive difference. It is conceivable that the process of students examining their interests, skills, and limits and comparing them to possible employment and education opportunities increases self-determination skills, which has been associated with post-school outcomes.

Importance of Effect Size in Understanding These Results

Meaningful effect size is often more important than statistical significance because significance values do not measure practical importance of the results (Thompson, 1998, 2006). The p-values of statistical tests are inherently flawed because it is next to impossible to draw a large enough random sample to reach needed power levels (Kehle et al., 2007). As far back as 1963, researchers argued against only using statistical significance testing for the basis of making decisions about the importance of scientific results (Johnson, 1999). The reliance on only statistical significance slows “the growth of scientific knowledge” (Schmidt & Hunter, 1997, p. 37), because the inclusion of only results that are statistically significant in the published literature discounts some studies with meaningful results that lack power (Kline, 2004).

This is one of those studies that lacked power because our analysis occurred at the district level since dependent variables were only available at the district level. Even with a high district response rate (including data from 40% of all districts across the state), power was less than optimal. However, effect sizes are not influenced by power issues resulting from smaller sample size (e.g., Cohen, 1988; Sullivan & Feinn, 2012) in the same ways as significance testing. Focusing interpretation on effect sizes in these cases can shed new light on previously incompletely understood problems (Wilkinson & The APA Task Force on Statistical Inference, 1999), such as in this case where differing levels of transition education practices resulted in meaningful effect sizes on Indicator 14 outcomes but not on Indicator 13 scores. These results need to be considered and the corresponding practices implemented to increase the likelihood of increased Indicators 13 and 14 results which ultimately appear to yield improved quality of post high school life for current and future students with disabilities.

Limitations and Needed Areas of Future Research

Every study has limitations, and we believe two major and a couple associated issues may limit the impact of the results of this study while simultaneously suggesting future research areas. First, a larger sample size would have enabled us to conclude more than we did from the results, including the ability to use factor analytic techniques to examine the underlying covariance structure of the survey, and including data from multiple states would have likely included different approaches to collecting Indicator 14 data. Moreover, the small sample size precluded us from employing regression techniques to determine the extent to which practice variables reported in this study predicted Indicators 13 and 14 results. Although we engaged with a substantial percentage of school districts in one state, the size of the state necessitated a small sample size. Future research needs to systematically replicate this study with a greater sample diversity and include information on district resources and administrative limitations that may have prevented the implementation of transition practices. Moreover, because all survey participants volunteered, these educators may have been more confident in their transition education practices than those who did not volunteer. Future studies should include investigating the practices of participating educators to ensure the survey reflects classroom practices.

Second, the number of students within the districts ranged from 100 to about 100,000 students. In addition to varied sample size, unmeasured student and district characteristics, such as more highly trained educators in more affluent districts or different administrative and funding resources, may have influenced usage of best transition education practices. We also had limited knowledge of the demographic composition of districts and could not determine if our sample adequately represented the entire state. Although our results were not affected by district size, socioeconomic variables, or number of minoritized students, this study did not control for these characteristics, which may limit interpretation.

Implications for Policy Makers and Conclusion

IDEA 2004 Part B secondary transition education indicators depict the extent to which states comply with transition education requirements within the IEP process, and the extent to which students with IEPs who left or graduated from high school transition into postsecondary education and employment. Examining the impact of transition education practices on Indicators 13 and 14 scores leads to numerous implications for policymakers at federal, state, and local levels.

First, this study found various levels of common transition education practices meaningfully impacted Indicator 14 postsecondary education and employment outcomes, but the transition practices themselves had no meaningful effect on Indicator 13 scores. Much more needs to be known prior to the next IDEA reauthorization regarding the impact and relationship of Indicators 13 and 14 to transition practice. This need necessitates federal encouragement and funding of additional experimental and correlational research to systematically replicate Erickson et al. (2014) and the present study, and to encourage new lines of research examining transition education indicators. Results of this study, such as increased involvement in IEP meetings and increased paid employment and money management experiences leading to higher post-school levels of employment and further education, should be considered in future federal policies and funding opportunities. Perhaps focused Office of Special Education Programs (OSEP) -funded special projects or the Institute for Education Sciences’ National Center for Special Education Research could provide specific research funding to examine secondary transition indicators, what influences them, and how evidence and research-based transition education practices impact transition indicators.

Second, most of the districts in this study report implementing only a few specific transition education interventions. Although we did not explicitly examine educators’ or administrators’ transition education knowledge, the lack of quality transition practices suggests limited proficiency. One conclusion is these educators may not have learned about transition education practices in their preparation programs, a conclusion also in the extant literature (e.g., Plotner & Simonsen, 2018). Williams-Diehm et al. (2018) found that only 35% of universities in their nationwide sample required a transition education course for special education teacher certification. Although outside the scope of the results presented in this study, it is possible that district-level limitations, such as lack of administrative support, or resource shortages, or limited student transition preparation time (e.g., Suk et al., 2020), contributed to the limited number of reported transition education interventions, which leads to the suggestion that greater administrative support and resources be allocated. Increased investigation of school- or district-level transition practices, using instruments such as the Secondary Transition Fidelity Assessment (Mazzotti et al., 2024), would enable administrators to provide additional support to transition professionals by helping them collect and evaluate data on their own transition programs.

We agree with Plotner and Simonsen (2018) that federal personnel preparation grants need to target funding to increase pre-service educators’ knowledge and experience with transition education practices. We further encourage district administrators to earmark resources to include teacher in-service opportunities or additional training for educators on increasing student involvement in IEP meetings, money management experiences, and ways to encourage paid employment experiences as part of educators’ required professional development experiences.

The measurement of Indicator 14 in this study, or the percentage of students who are employed and/or enrolled in further education upon leaving high school, is at the crux of transition education. This study identified the extent to which school districts in one state used transition education practices and then analyzed the impact those practices had upon districts’ Indicator 13 scores and Indicator 14 outcomes. Results show that when students are entrusted with higher levels of involvement in their IEP meetings and have higher levels of paid employment and money management experience, they experience increased employment and enrollment in further education. District personnel and state and federal policy makers should direct their attention to these results and ensure the conditions under which these students are educated exhibit the necessary features (Mazzotti et al., 2024) that lead to these positive post-school outcomes.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

Funding from the U.S. Department of Education, Office of Special Education Leadership Grant #H325D050031 partially supported the completion of the research study.

ORCID iD

Maeghan N. Hennessey

References

Carter

E. W.

Austin

Trainor

A. A.

(2012). Predictors of postschool employment outcomes for young adults with severe disabilities. Journal of Disability Policy Studies, 23(1), 50–63. https://journals-sagepub-com.ezproxy.lib.ou.edu/doi/pdf/10.1177/1044207311414680. https://doi.org/10.1177/1044207311414680

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence Erlbaum.

Enayati

Karpur

(2019). Impact of participation in school-to-work programs on postsecondary outcomes for youth with disabilities from low-income families. Journal of Disability Policy Studies, 29(4), 235–244. https://journals-sagepub-com.ezproxy.lib.ou.edu/doi/pdf/10.1177/1044207318789419. https://doi.org/10.1177/1044207318789419

Erickson

A. S. G.

Noonan

P. M.

Brussow

J. A.

Giplin

B. J.

(2014). The impact of Indicator 13 compliance on postsecondary outcomes. Career Development for Exceptional Individuals, 37(3), 161–167. https://doi.org/10.1177/2165143413481497

Gerber

P. J.

Batalo

C. G.

De Arment

S. T.

(2014). An analysis of state data collection protocols for measuring postschool outcomes for students with disabilities. Career Development and Transition for Exceptional Individuals, 37(2), 97–105. https://doi.org/10.117/2156143413481380

Haber

M. G.

Mazzotti

V. L.

Mustian

A. L.

Rowe

D. A.

Bartholomew

A. L.

Test

D. W.

Fowler

C. H.

(2016). What works, when, for whom, and with whom: A meta-analytic review of predictors of postsecondary success for students with disabilities. Review of Educational Research, 86, 123–162. https://doi.org/10.3102/0034654315583135

Individuals with Disabilities Education Improvement Act. (2004). Pub. L. 108-446, §1.

Johnson

D. H.

(1999). The insignificance of statistical significance testing. Journal of Wildlife Management, 63(3), 763–772. https://doi.org/10.2307/3802789

Joshi

G. S.

Bouck

E. C.

Maeda

(2012). Exploring employment preparation and postschool outcomes for students with mild intellectual disability. Career Development and Transition for Exceptional Individuals, 35, 97–107. https://doi.org/10.1177/2165143412433822

10.

Zeng

Landmark

L. J.

(2017). Self-determination and academic success of students with disabilities in postsecondary education: A review. Journal of Disability Policy Studies, 28(3), 180–189. https://doi.org/10.1177/1044207317739402

11.

Kehle

T. J.

Bray

M. A.

Chafouleas

S. M.

Kawano

(2007). Lack of statistical significance. Psychology in the Schools, 44(5), 417–422. https://doi.org/10/1002/pits.20233

12.

Kline

R. B.

(2004). Beyond significance testing: Reforming data analysis methods in behavioral research. American Psychological Association.

13.

Kohler

P. D.

Gothberg

J. E.

Fowler

Coyle

(2016). A taxonomy for transition programming 2.0: A model for planning, organizing, and evaluating transition education, services, and programs. Western Michigan University. https://transitionta.org/wp-content/uploads/docs/NTACT-C_TaxonomyforTransition.pdf

14.

Konrad

Fowler

C. H.

Walker

Test

Wood

W. M.

(2007). Effects of self-determination interventions on the academic skills of students with learning disabilities. Learning Disabilities Quarterly, 30(2), 270–285. https://doi.org/10.2307/30035545

15.

Lakens

(2014, June 7). Calculating confidence intervals for Cohen’s d and eta-squared using SPSS, R, and Stata. The 20% Statistician. http://daniellakens.blogspot.com/2014/06/calculating-confidence-intervals-for.html

16.

Lee

Wehmeyer

M. L.

Palmer

S. B.

Soukup

J. H.

Little

T. D.

(2008). Self-determination and access to the general education curriculum. The Journal of Special Education, 42(2), 91–107. https://doi.org/10.1177/0022466907312354

17.

Madaus

J. W.

Gelbar

Dukes

L. L.

Taconet

Faggella-Luby

(2021). Are there predictors of success for students with disabilities pursuing postsecondary education? Career Development and Transition for Exceptional Individuals, 44(4), 191–202. https://doi.org/10.1177/2165143420976526

18.

Martin

J. E.

Marshall

L. H.

Maxson

Jerman

(2016). Self-directed IEP. http://www.ou.edu/content/education/centers-and-partnerships/zarrow/choicemaker-curriculum/self-directed-iep.html

19.

Martin

J. E.

Van Dycke

J. L.

Christensen

W. R.

Greene

B. A.

Gardner

J. E.

Lovett

D. L.

(2006). Increasing student participation in IEP meetings: Establishing the Self-Directed IEP as an evidenced-based practice. Exceptional Children, 72(3), 299–316. https://doi.org/10.1177/001440290607200303

20.

Mazzotti

V. L.

Morningstar

M. E.

Lombardi

Raley

S. K.

Rowe

D. A.

Poppen

Test

D. W.

(2024). Conceptualizing and developing the secondary transition fidelity assessment to support transition programming and practice. Career Development and Transition for Exceptional Individuals, 47(2), 106–118. https://doi.org/10.1177/21651434231163498

21.

Mazzotti

V. L.

Rowe

D. A.

Kwiatek

Vofft

Chang

Fowler

C. H.

Poppen

Sinclair

Test

D. S.

(2021). Secondary transition predictors of postschool success: An update to the research base. Career Development and Transition for Exceptional Individuals, 44(1), 47–64. https://doi.org/10.1177/2165143420959793

22.

McConnell

A. E.

Martin

J. E.

Juan

C. Y.

Hennessey

M. N.

Terry

R. A.

el-Kazimi

N. A.

Pannells

T. C.

Willis

D. M.

(2012). Identifying nonacademic behaviors associated with post-school employment and education. Career Development and Transition for Exceptional Individuals, 36(3), 174–187. https://doi.org/10.1177/2165143412468147

23.

National Technical Assistance Center on Transition. (n.d.). Checklists. https://transitionta.org/category/resource/checklists

24.

National Technical Assistance Center on Transition. (2016). Predictors of post school success level of evidence chart. https://transitionta.org/predictors-of-post-school-success/

25.

Papay

C. K.

Bambara

L. M.

(2014). Best practices in transition to adult life for youth with intellectual disabilities. Career Development and Transition for Exceptional Individuals, 37(3), 136–148. https://doi.org/10.1177/2165143413486693

26.

Plotner

A. J.

Simonsen

M. L.

(2018). Examining federally funded secondary transition personnel preparation programs. Career Development and Transition for Exceptional Individuals, 41(1), 39–49. https://doi.org/10.1177/2165143417742138

27.

Powers

K. M.

Gil-Kashiwabara

Geenen

S. J.

Powers

L. E.

Balandran

Palmer

(2005). Mandates and effective transition planning practices reflected in IEPs. Career Development for Exceptional Individuals, 28(1), 47–59. https://doi.org/10.1177/08857288050280010701

28.

Rowe

D. A.

Alverson

C. Y.

Unruh

D. K.

Fowler

C. H.

Kellems

Test

D. W.

(2015). A Delphi Study to operationalize evidence-based predictors in secondary transition. Career Development and Transition for Exceptional Individuals, 38(2), 113–126. https://doi.org/10.1177/2165143414526429

29.

Rowe

D. A.

Mazzotti

V. L.

Fowler

C. H.

Test

D. W.

Mitchell

V. J.

Clark

K. A.

Holzberg

Owens

T. L.

Rusher

Seaman-Tullis

R. L.

Gushanas

C. M.

Castle

Chang

Voggt

Kwiatek

Dean

J. C.

(2021). Updating the secondary transition research base: Evidence- and research-based practices in functional skills. Career Development and Transition for Exceptional Individuals, 44(1), 28–46. https://doi.org/10.1177/2165143420958674

30.

Rusch

F. R.

Hughes

Agran

Martin

J. E.

Johnson

J. R.

(2009). Toward self-directed learning, post-high school placement, and coordinated support: Constructing new transition bridges to adult life. Career Development for Exceptional Individuals, 32(10), 53–59. https://doi.org/10.1177/0885728809332628

31.

Schmidt

F. L.

Hunter

J. E.

(1997). Eight common but false objections to the discontinuation of significance testing in the analysis of research data. In Harlow

L. L.

Mulaik

S. A.

Steiger

J. H.

(Eds.), What if there were no significance tests? (pp. 37–64). Lawrence Erlbaum.

32.

Seong

Wehmeyer

M. L.

Palmer

S. B.

Little

T. D.

(2015). Effects of the Self-Directed Individualized Education Program on self-determination and transition of adolescents with disabilities. Career Development and Transition for Exceptional Individuals, 38(3), 132–141. https://doi.org/10.1177/2165143414544359

33.

Shogren

K. A.

Wehmeyer

M. L.

Palmer

S. B.

Rifenbark

G. G.

Little

T. D.

(2015). Relationships between self-determination and postschool outcomes for youth with disabilities. Journal of Special Education, 48(4), 89–103. https://doi.org/10.1177/0022466913489733

34.

Suk

A. L.

Martin

J. E.

McConnell

A. E.

Biles

T. L.

(2020). States decrease their required secondary transition planning age: Federal policy must change. Journal of Disability Policy Studies, 31(2), 112–118. https://doi.org/10.1177/1044207320915157

35.

Sullivan

G. M.

Feinn

(2012). Using effect size—Or why the p value is not enough. Journal of Graduate Medical Education, 4(3), 279–282. https://doi.org/10.4300/JGME-D-12-00156.1

36.

Test

D. W.

Fowler

C. H.

Richter

S. M.

White

Mazzotti

Walker

A. R.

Kohler

Kotering

(2009). Evidence-based practices in secondary transition. Career Development for Exceptional Individuals, 32(2), 115–128. https://doi.org/10.1177/0885728809336859

37.

Thompson

(1998). Statistical significance and effect size reporting: Portrait of a possible future. Research in the Schools, 5(2), 33–38. https://files.eric.ed.gov/fulltext/ED429090.pdf#page=36

38.

Thompson

(2006). Foundations of behavioral statistics: An insight-based approach. The Guilford Press.

39.

Vacha-Haase

Nilsson

J. E.

Reetz

D. R.

Lance

T. S.

Thompson

(2000). Reporting practices and APA editorial policies regarding statistical significance and effect size. Theory & Psychology, 10(3), 413–425. https://doi.org/10.1177/0959354300103006

40.

Ward

J. T.

(1963). Hierarchical grouping to optimize and objective function. Journal of the American Statistical Association, 58(301), 236–244. https://www.jstor.org/stable/2282967. https://doi.org/10.1080/01621459.1963.10500845

41.

Wehman

Sima

A. P.

Ketchum

West

M. D.

Chan

Luecking

(2015). Predictors of successful transition from school to employment for youth with disabilities. Journal of Occupational Rehabilitation, 25(2), 323–334. https://www-proquest-com.ezproxy.lib.ou.edu/docview/1681446511/fulltextPDF/30FFE26AA01E4B26PQ/1?accountid=12964. https://doi.org/10.1007/s10926-014-9541-6

42.

Wei

Wagner

Hudson

J. W.

Javitz

(2016). The effect of transition planning participation and goal-setting on college enrollment among youth with autism spectrum disorders. Remedial and Special Education, 37(1), 3–14. https://doi.org/10.1177/0741932515581495

43.

Wilkinson

, & Task Force on Statistical Inference, American Psychological Association, Science Directorate. (1999). Statistical methods in psychology journals: Guidelines and explanations. American Psychologist, 54(8), 594–604. https://doi.org/10.1037/0003-066X.54.8.594

44.

Williams-Diehm

K. L.

Rowe

D. A.

Johnson

M. C.

Guilmeus

J. F.

(2018). A systematic analysis of transition coursework required for special education licensure. Career Development and Transition for Exceptional Individuals, 41(1), 16–26. https://doi.org/10.1177/2165143417742404