Impact of an IEP Goal Builder on Social Goals for Autistic Students

Abstract

This pilot study examined (a) the perceived barriers to creating high-quality social and emotional learning (SEL) IEP goals for autistic students without intellectual disabilities, and (b) the impact of using a data-driven SEL IEP goal builder—a key component of the Ivymount Social Cognition Instructional Package (IvySCIP)—on the quality of SEL IEP goals. Based on data from 40 instructor/student dyads, we found the following: First, based on phone interviews, instructors identified a number of significant barriers to the creation of SEL IEP goals for this population, including lack of ability to reliably assess students’ SEL strengths and needs, lack of a goal builder/goal bank, and lack of comprehensive, data-driven support for generating SEL goals and tracking progress toward goal mastery. Interviewees agreed that IvySCIP effectively addressed these barriers. Second, based on a comparison of pre-intervention SEL IEP goals and those created using the IvySCIP goal builder, the quality of instructors’ IEP goals appeared to improve after introduction of the goal builder, with post-intervention goals significantly more likely to include settings, givens, specific/demonstrable behavior being targeted, and measurement criteria to ensure meaningful progress monitoring. Post-intervention goals also drew more equitably from a range of SEL domains.

Keywords

autism social cognition social and emotional learning assessment IEP goals

Social and emotional learning (SEL) challenges are one of the key difficulties autistic students face, including autistic students without intellectual disabilities (ASw/oID, see Author Note) who are estimated to make up approximately two thirds of the autistic population (Diagnostic and Statistical Manual of Mental Disorders–Fifth Edition [DSM-5; American Psychiatric Association, 2013]; Maenner et al., 2020). These include significant challenges across all five of the Center for Academic and Social and Emotional Learning [CASEL] domains: (1) self-awareness (e.g., the ability to identify one’s strengths and weaknesses, and to advocate for needed supports), (2) self-management (e.g., emotion regulation), (3) responsible decision-making (e.g., social problem-solving, goal setting, and other executive functioning behaviors), (4) social awareness (e.g., theory of mind and perspective taking), and (5) relationship skills (e.g., the ability to navigate interpersonal give and take during conversation and/or play).

Social and emotional learning instruction has been linked to improved academic outcomes, increased pro-social interactions with peers, and reductions in challenging behaviors (Corcoran et al., 2018; Durlak et al., 2011). Most special educators recognize the importance of SEL instruction for their students with disabilities (Espelage et al., 2016)—including ASw/oID (Cappadocia & Weiss, 2011). For example, in a survey of 70 special education teachers about SEL Individualized Education Program (IEP) goals, researchers found that 93% believed SEL should be an important component of in-school experience, and 92% agreed that it was their responsibility to address the SEL needs of their students (Bridgeland et al., 2013). Another study found that teachers who reported both greater comfort teaching SEL and greater commitment to SEL training, also reported they felt a greater sense of accomplishment through teaching (Brackett et al., 2012).

SEL and the Importance of High-Quality IEP Goals

But does a commitment to SEL translate into high-quality SEL IEP goals? In spite of the fact that teachers seem to recognize the value of SEL, a survey found that the majority of participating teachers reported a lack of confidence in their ability to write effective IEPs for students with special needs, especially autistic students (Busby et al., 2012), and a growing body of literature demonstrates some of the critical shortcomings related to educators’ IEP goal writing skills.

A study of 75 students’ IEPs identified misalignment between students’ SEL present level of performances (PLOP) and their identified SEL services, with 93.3% of the students having social skills services recommended in their psychological evaluations, but only 41% receiving any type of in-school counseling, social skills, or social work services (Gelbar et al., 2018). Similarly, a study by Findley et al. (2022) found that few transition-age students’ IEPs included SEL goals or demonstrated alignment between autistic students’ PLOP and their IEP goals. This disparity between identified SEL needs and SEL IEP goals is intensified in rural environments. For example, in a sample of 126 rural autistic students, researchers found that 75% of their educational records indicated social, emotional, or behavioral needs, but only 18% of IEPs included social, emotional, or behavioral goals that addressed the specific needs identified in their PLOP (Hott et al., 2019). Lucas et al. (2014) argue that the key to supporting development of high-quality IEP goals is creating a deliberate link between functional assessment of student skills and goal development. In an article on avoiding substantive errors in IEP goal development, Yell and colleagues (2016) suggested that one of the most common errors of IEP goal developers was failure to do just this (i.e., conduct a complete and individualized assessment of students’ strengths and needs and link assessment to goal development). This can lead to a denial of a free, appropriate public education (FAPE) and violation of the Individuals with Disabilities Education Act (IDEA; Yell et al., 2016).

Another study suggested that teachers were more likely to define SEL in terms of classroom discipline and work habits as opposed to defining SEL in terms of established CASEL domains like emotional regulation and social interaction (Bridgeland et al., 2013). These misconceptions about what SEL is leads to IEP goals that focus on behaviors such as following rules and attending class, as opposed to more inter- and intra-personal skills like initiating conversation or play, taking turns, and coping with frustration (Bridgeland et al., 2013).

Related to this, reviews of IEP goal quality have shown the majority of educators write goals that do not clearly specify target behaviors and/or allow for data collection and evaluation (Diehm, 2017). For example, one study of IEPs provided by 135 special educators found that goals were generally poorly written, especially in terms of measurability with fewer than 10% of goals including quantitative criteria to measure performance (Sanches-Ferreira et al., 2013). Another study found that out of 70 students with disabilities, whose IEPs included social, emotional, or behavioral goals, only 4% of these goals both addressed students’ needs and included progress monitoring methods (Hott et al., 2019). While recent research on IEPs of autistic students is lacking, in an older study of 35 teachers of autistic students, researchers found that only 1.9% of goals included a method of goal measurement (Ruble et al., 2010). Furthermore, only 41% of goals were judged as able to be measured in behavioral terms, and only 39% of goals included the conditions under which the behavior was to occur (i.e., settings and givens). Ruble et al. (2010) concluded that the IEPs of students with autism spectrum disorder (ASD) do not meet the requirements of IDEA, and that measurability of IEP objectives appeared to be one of the greatest areas of need—in particular, specific criteria for measuring goal attainment.

These findings are troubling because research suggests that weak IEP goals are correlated to poor student outcomes (Ruble & McGrew, 2013). The good news, however, is that when educators are supported in writing more robust IEP goals (i.e., operationally defined IEP goals and objectives based on students’ identified needs), student learning is enhanced. Research indicates that students’ needs are more likely to be met when assessment results, IEP goals, and approaches to intervention are directly linked to one another. For example, a study by Ruble and McGrew (2013) found that training and support for developing IEP goals resulted in significantly higher quality IEP goals. Similarly, a study by Boavida et al. (2014) found that training early childhood interventionists on goal writing resulted in significantly improved IEP goal, and a study by Russo-Campisi (2020) found that online instruction on goal also resulted in higher quality goals.

Goal Components and Electronic IEP Goal Banks

A growing body of literature focuses on ensuring that IEP goals include all key components of a high-quality goal (i.e., are based on student’s identified strengths and needs, and include settings and givens, specific/demonstrable behavior, and measurement criteria; Rowland et al., 2015; Yell et al., 2016). For example, Rowland et al. (2015) recommend that IEP developers consider whether goals are focused/precise, measurable, and student centered, and Yell and colleagues (2016) stress that goals must contain all necessary components—that is, fully operationalized behaviors and measurement criteria—to ensure IEP developers meet the procedural and substantive requirements of IDEA.

Several articles suggest that electronic IEP software that includes an electronic goal bank can help with this process (More & Hart, 2013; More & Hart-Barnett, 2014). Electronic IEP software can result in IEPs that demonstrate a clear link between assessed needs and the skills students are excepted to achieve, are easy to read, match students’ PLOP, and produce attainable, observable, and measurable goals (More & Hart, 2013; More & Hart-Barnett, 2014). Some of the advantages of an IEP goal bank include reducing the time it takes for instructors to write goals, allowing modification of goals, and including default features that ensure IEP goals include all necessary components for compliance (More & Hart-Barnett, 2014). Unfortunately, many electronic IEP goal banks focus exclusively on academic goals and lack SEL goals (More & Hart-Barnett, 2014).

IvySCIP Goal Builder and Purpose of Study

Ivymount Social Cognition Instructional Package (IvySCIP) is a comprehensive online tool designed to support data-driven SEL instruction—from assessment of strengths and needs, though identification of instructional priority areas, SEL IEP goal development, and progress monitoring. IvySCIP was specifically developed by Ivymount School and Programs in Rockville, Maryland, and 3C Institute in Durham, North Carolina, in part to address the gaps described earlier by

providing data-driven support for the development of SEL IEP goals;

supporting the identification of instructional priority areas (i.e., helping users identify “where to begin”);

supporting development of specific, measurable, high-quality goals; and

offering a customizable SEL IEP goal bank.

As part of a 20-week pilot study, authors collected extensive quantitative and qualitative data to test the impact of IvySCIP on students and instructors. Our first article based on these data found a correlation between IvySCIP use and increases in students’ SEL outcomes as measured by multiple standardized tools, and began to establish the psychometric properties of the IvySCIP assessment tool (Müller et al., 2022). For the purposes of the present article, we sought to answer the following four questions using a subset of our data:

What are the key issues in developing quality SEL IEP goals that participating instructors face?

What are the benefits of using IvySCIP for addressing these issues?

Are SEL IEP goals generated using the IvySCIP goal builder of higher quality than SEL IEP goals generated prior to its introduction?

Does creation of data-driven IEP goals using IvySCIP result in more equitable distribution of goals across SEL domains?

We also established social validity of the IvySCIP SEL IEP goal builder via post-intervention product surveys in which participants evaluated the overall usefulness and value of the IvySCIP goal builder and the data-driven way in which goals are created.

Method

Participants

We recruited public- and private-school instructors, including special education teachers, behavior specialists, school mental health providers, and speech-language pathologists across four U.S. states (i.e., Maryland, North Carolina, Pennsylvania, and Virginia). To participate, instructors verified that they were currently providing instruction to and writing IEP goals for at least one eligible student. An eligible student was defined as (a) enrolled in kindergarten through fifth grade, (b) identified as having autism through school or psychological testing, as well as through parent report, (c) had an intelligence quotient (IQ) of 75 or higher, (d) demonstrated SEL challenges in one or more CASEL domains, (e) either attended a regular education classroom for 40% or more of the school day or, if receiving instruction in a self-contained classroom, was within one grade level of functioning (i.e., above or below grade level) with supports, (f) demonstrated at least four word phrased speech, and (g) had been working with the instructor for at least 1 month.

More than 120 instructors completed the online eligibility form, and the first 60 eligible instructors were selected for participation in the study. Fifty-one instructor/student dyads ultimately participated in the pilot study. For the purposes of the present sub-analysis, we were able to secure copies of current IEPs for 40 participating students, review IEP goals generated using the IvySCIP goal builder by the 40 participating instructors paired with each of the students, and conduct phone interviews with 37 of the participating instructors. Reasons for reduced sample size in terms of IEPs and phone interviews included study attrition toward the end of the academic year, missing records, and/or scheduling difficulties. In terms of student demographics, there were 32 males, seven females, and one gender unreported; two Asian, three Black/African American, one Native Hawaiian/Pacific Islander, 25 White, five reporting more than one race, and four other/unreported, and age range was 5–10 years. In terms of instructor demographics, there were two males and 38 females; two Asian, 35 White, one reporting more than one race, and two other/unreported; and age range was 23–61 years. Nine had a bachelor’s degree and 31 had a master’s degree. At the time of the study, 16 instructors had been teaching for 11 or more years, nine instructors had been teaching for between 6 and 10 years, eight instructors had been teaching for between 3 and 5 years, and seven had been teaching for 2 years or less.

Study Design

This pilot study was designed as a mixed methods study, combining quantitative and qualitative measures, including review of students’ extant IEP goals, comparison of extant goals with goals created using IvySCIP, and end-of-intervention interviews with participating instructors. The data used for this pilot study were part of a data set from a larger 20-week study looking at the overall impact on instructors and students of using IvySCIP to support SEL.

Data Collection

IEP goals

In addition to creating two SEL goals using the IvySCIP goal builder, participating instructors were asked to submit copies of students’ current IEPs so that authors could compare (a) the quality of SEL goals and (b) the proportional representation of SEL domains before and after use of the goal builder. Authors included only those participants for whom IEPs were submitted, resulting in data for 40 instructors with a collective total of 109 goals created prior to using IvySCIP (ranging from one to seven SEL goals per student) and a collective total of 80 goals created using the IvySCIP goal builder (two SEL goals per student).

At the start of the study, instructors attended a 1-hr training webinar on IvySCIP functions and flow, and watched an implementation training video with detailed instruction on how to use the separate features of the IvySCIP, including creation of goals using the IvySCIP goal builder. To enhance fidelity of implementation, instructors had to complete a post-test after they viewed the training video to ensure they mastered all key video content. Instructors could only begin intervention after they achieved 90% accuracy on post-test questions.

Prior to using the IvySCIP goal builder, instructors completed the IvySCIP strengths and needs assessment, a 113-item social skills assessment. Based on assessment results, IvySCIP automatically generates a list of instructional priority areas. Based on their knowledge of the student, instructors can choose one or more of the instructional priority areas they want to address, and build an IEP using the IvySCIP goal builder. The customizable goal builder helps ensure that the goal is data-driven (based on needs identified via the strengths and needs assessment), and includes all key components of a high-quality goal (i.e., setting and givens, specific/demonstrable behavior, and measurement criteria).

End-of-study interviews

At the conclusion of the 20-week pilot study, 37 instructors participated in end-of-study phone interviews. These interviews lasted between 25 and 45 min, and addressed a wide range of issues related to the provision of SEL instruction for ASw/oID prior to IvySCIP, as well as the impact of IvySCIP on SEL instruction and IEP goal development. Interviews were audiotaped and transcribed verbatim either at the time of the interview or after the interview.

Data Analysis

IEP goals

Goal components

Social and emotional learning goals were coded for absence/presence of the following components: (a) setting (e.g., small group, general education classroom), (b) given(s) (e.g., social story, modeling of behavior), (c) specific/demonstrable targeted behavior (e.g., greeting a peer in the hallway, sustaining conversation over four turns), and (d) measurement criteria (e.g., demonstrating behavior in 8 out of 10 trials across 1 week). Authors then calculated average number of goals including each of these components created both before introduction of the IvySCIP goal builder, and using the tool. Differences in mean totals for each condition were calculated using a series of t tests. Differences were found to be significant at the .05 level. Effect sizes were calculated using Cohen’s d.

Inter-rater reliability was assessed by having the second author code a random sample of the goals being analyzed (i.e., at least 30% of IEP goals developed before introduction of the IvySCIP and 30% of goals developed using the IvySCIP goal builder). Inter-rater reliability was calculated by dividing the total number of agreements between the authors by the total number of agreements plus the total number of disagreements between the two authors, and multiplying by 100. Inter-rater reliability was 98% for setting, 100% for givens, 98% for specific targeted behavior, and 98% for clearly specified measurement criteria.

SEL domain representation

Authors counted the number of SEL goals that fell within each of the following domains: (a) social interaction, (b) self-awareness and advocacy, (c) emotion regulation, (d) executive skills and critical thinking, (e) self-care, (f) reduction of challenging behaviors, and (g) other. Authors then calculated average percentage of goals representing each of these domains created both before use of the IvySCIP goal builder and after introduction of the tool. Differences in percentages of goals falling within each domain were calculated using a series of paired samples t tests. Differences were found to be significant at the .05 level. Effect sizes were calculated using Cohen’s d.

Post-intervention interviews

Analysis of interview transcripts took place in several phases and was conducted systematically using methods described by Miles and Huberman (1994). Initially, the two first authors independently engaged in data reduction by reviewing transcripts of interviews and labeling/coding any comments that pertained to (a) barriers to SEL IEP goal creation for ASw/oID and/or (b) ways in which IvySCIP benefited SEL IEP goal creation for ASw/oID. Authors met weekly over the course of 2 months to discuss and consolidate findings and establish a coding tree. Any differences of opinion were resolved by returning to the data and establishing, via consensus, what the data were actually “saying.” One of the first authors then went back and coded all transcripts accordingly, making minor modifications to the coding tree as needed. Major themes were defined as those mentioned by 10 to 20 interviewees. Minor themes were defined as those mentioned by between five and nine interviewees. A third and final layer of analysis were intended to generate an explanatory framework for study findings. Whereas the first two phases were mainly descriptive in nature, and focused solely on the data, this final step (see section “Discussion”) integrated ideas from the literature on SEL IEP goals and students with ASD.

In designing the qualitative component of this study, authors sought to meet the credibility indicators for qualitative research (Brantlinger et al., 2005), including the following:

Investigator triangulation—both first authors separately analyzed and coded all interview transcripts;

Researcher reflexivity—all four authors identified sources of potential bias (see next paragraph’s statement regarding transparency);

Collaborative work—authors met weekly as a team to discuss research design, data collection, and data analysis; and

Audit trail—the two first authors met regularly to compare and contrast findings/observations and kept notes as the coding tree evolved over time.

In an effort to be transparent, authors felt it was important to self-disclose the following information about potential biases: Three of the authors of this study were also authors of the IvySCIP. All four authors were inclined to believe that IvySCIP would prove to be beneficial in supporting the development of high-quality SEL IEP goals for ASw/oID.

Findings

IEP goals

Goal components

A comparison of SEL IEP goals before and after use of the IvySCIP goal builder found the following: (a) Information on setting was present in only 27.5% of goals prior to introduction of the goal builder, but present in 92.5% of goals after introduction; (b) information on givens was present in only 36.7% of goals prior to introduction of the goal builder, but present in 100% of goals after; (c) targeted behaviors were clearly specified in 83.5% of goals prior to introduction of the goal builder, but clearly specified in 100% of goals after; and (d) measurement criteria were clearly specified in 46.8% of goals prior to introduction of the goal builder, but clearly specified in 97% of goals after introduction (see Table 1). These changes in each of the four categories were statistically significant at the .001 levels after the introduction of the goal builder, with the most significant change occurring in the givens category. Effect sizes were large for setting and givens, medium/large for measurement criteria, and small/medium for target skill/behavior.

Table 1.

Presence of Goal Components Before and After Introduction of IvySCIP.

Goal component	Before IvySCIP		After IvySCIP		Difference		t	Effect size^a
Goal component	M	(SD)	M	(SD)	M	(SD)	t	Effect size^a
Setting	0.28	(0.45)	0.93	(0.26)	0.61	(0.49)	10**	1.2
Given(s)	0.37	(0.48)	1.00	(0.00)	0.65	(0.48)	11**	1.4
Measure	0.48	(0.50)	0.97	(0.17)	0.42	(0.53)	6.5**	0.79
Skill/behavior	0.83	(0.37)	1.00	(0.00)	0.18	(0.39)	3.8**	0.46

Note. IvySCIP = Ivymount Social Cognition Instructional Package.

ES > .02 = small, > .05 = medium, > .08 = large.

p < .001.

The following are examples of typical goals created prior to introduction of the IvySCIP goal builder: “Will identify when I am feeling upset or frustrated and use an appropriate strategy. After encountering an upsetting situation I will use a strategy and return to the original task” (Self-Awareness) or “I will use low and slow words to describe when I am feeling overwhelmed or upset” (Relationship Skills). The following much more complete goals, however, were typical of those created after the IvySCIP goal builder was introduced: “In a small group setting, given visual and verbal cues, student will choose one strategy from a menu of choices to support a social challenge, and use a teacher-made template to reflect on how successful the strategy was during 4 out of 5 observed opportunities across three consecutive sessions” (Self-Awareness & Advocacy), and “In a general education setting, given a visual, a script, and role-play practice, student will calmly (e.g., conversational tone/volume and calm body) implement a pre-determined strategy to resolve a difficult interaction with a peer in 70% of observed opportunities across three consecutive weeks” (Executive Skills & Critical Thinking).

SEL domain representation

Based on comparison of goals developed before and after introduction of the data-driven IvySCIP goal builder, we found that pre-intervention goals tended to cluster primarily in the domains of social interaction (58.7%) and executive skills and critical thinking (22%), with fewer goals falling into the categories of self-awareness and advocacy (5.5%), emotion regulation (8.25%), and none falling into the domain of self-care (see Table 2). A few additional pre-intervention goals were not about skill building at all, and focused instead on reduction of challenging behaviors (2.75%). Several more were labeled as SEL goals but did not address generally recognized SEL domains (2.75%). Post-intervention goals, however, were more evenly spread out across the five SEL domains of social interaction (35%), executive skills and critical thinking (25%), self-awareness and advocacy (17.5%), emotion regulation (20%), and self-care (2.5%). The changes in number of goals for both the social interaction and self-awareness and advocacy domains were statistically significant (p < .05). While not reaching statistical significance, data also trended in a positive direction for executive skills/critical thinking, and emotion regulation. Effect sizes were medium for the reduction in social interaction goals, and small for increases in self-awareness/self-advocacy, emotional regulation, and self-care goals. Effect sizes were also small in terms of the reduction in goals targeting elimination of challenging behaviors and “other” goals that did not appear to fall into any of the CASEL domains.

Table 2.

Domains of IEP Goals Set Before and After Introduction of IvySCIP.

Domain	IvySCIP		Difference		t	Effect size^a
Domain	Before (%)	After (%)	M	(SD)	t	Effect size^a
Social interaction (SI)	58.7	35	0.90	(1.45)	3.94**	0.62
Self-awareness and advocacy (SAA)	5.5	17.5	0.20	(0.57)	2.24*	0.35
Emotion regulation (ER)	8.25	20	0.18	(0.64)	1.74	0.28
Executive skills & critical thinking	22	25	0.10	(1.01)	0.60	0.10
Self-care	0	2.5	0.05	(0.22)	1.43	0.23
Challenging behavior	2.75	0	0.08	(0.27)	1.78	0.30
Other	2.75	0	0.08	(0.27)	1.78	0.30

Note. IvySCIP = Ivymount Social Cognition Instructional Package.

ES > .02 = small, > .05 = medium, > .08 = large.

p < .05. **p < .001.

End-of-Study Interviews

Barriers

Two major and six minor themes emerged relating to key barriers to creating SEL IEP goals for ASw/oID (see Table 3). Major themes included the following: (a) lack of a comprehensive system—typical comments included, “We don’t have a system for IEP goals,” and “[We’ve got] nothing holistic, nothing comprehensive, only pieces of the puzzle” and (b) lack of data-driven goal setting—typical comments included, “We’re generating all our goals off of observation, and it’s very time consuming,” and “There are times that if we didn’t have the evidence, we might be more subjective rather than objective with data. Minor themes included (a) lack of teacher knowledge/preparedness—typical comments included, “A lot of special education teachers and other professionals struggle with goals,” (b) inability to prioritize—typical comments included, “There are too many needs [and] prioritizing where to start is the biggest challenge,” (c) lack of goal bank—typical comments included, “The way we do IEP goals, we pull from wherever, and not a set goal bank,” (d) lack of a way to ensure all components of goal are present—typical comments included, “It’s sometimes hard to find measurable goals . . . I have difficulties finding the exact wording, so I and others know what we’re collecting data on,” (e) lack of resources for “teaching to” goals and monitoring achievement—typical comments included, “I see all the people with these crazy goals, but no way to support the student to make progress with this goal,” (f) lack of resources to support new teachers and/or share with parents/colleagues in terms of SEL goals—typical comments included, “I think when I was a first, second, or third year [teacher] I didn’t know how to encompass it all, because SEL goals are not something that’s given to you,” and “One of the difficulties is explaining [SEL] to parents.”

Table 3.

Summary of Major and Minor Interview Themes.

Benefits	Barriers
Goals informed by data-driven assessment of students’ strengths and needs	Lack of a comprehensive system (major, n = 10)
• Provides assessment/data-driven support for SEL goal development (major, n = 32)	Lack of data-driven goal setting (major, n = 11)
	Lack of teacher knowledge/preparedness (minor, n = 7)
• Helps prioritize areas of SEL need (major, n = 17)	Inability to prioritize (minor, n = 9)
• Provides a comprehensive picture of students’ SEL strengths/needs (major, n = 22)	Lack of a goal bank (minor, n = 9)
• Supports goal development for new/more challenging students (minor, n = 6)	Lack of way to ensure all components of a goal are present (minor, n = 5)
High-quality goal builder and goals	Lack of resources for “teaching to” goal and monitoring achievement (minor, n = 9)
• Positive response to goal builder (major, n = 36)
• Creates measurable, high-quality goals containing all necessary components (major, n = 17)	Lack of SEL goal resources to support new teachers and/or share with parents/colleagues (minor, n = 9)
• Provides a customizable goal bank (major, n = 12)
Comprehensive system linking goals to other resources
• Provides a comprehensive system for developing and implementing goals (major, n = 23)
• Links goals to appropriate curricular materials (major, n = 12)
• Links goals to progress monitoring resources (major, n = 13)
Benefit to teachers
• Easy to use and saves time (major, n = 14)
• Increases teacher confidence (major, n = 18)
• Provides information to share with colleagues (major, n = 17)
• Supports new teachers or teachers unfamiliar with crafting SEL goals (major, n = 10)
• Provides information to support/share with parents (minor, n = 8)
Benefit to students
• Useful with a broad range of students (major, n = 16)

Note. SEL = Social and Emotional Learning.

Benefits

Major and minor themes relating to key benefits of using IvySCIP as part of the SEL IEP goal development and tracking process fell into five overarching categories.

The first category was goals informed by data-driven assessment of students’ strengths and needs, which included the following three major themes: (a) provides assessment/data-driven support for SEL goal development—typical comments included, “I really liked the direct link from assessment to prioritizing goal areas. Many times, the advantage of that is it can provide some evidence to [support selection of] instructional priority. There are times that if we didn’t have the evidence, we might be more subjective rather than objective with data. I like that part of the IvySCIP protocol, that it was really data driven from assessment through the whole process,” (b) helps prioritize areas of SEL need—typical comments included, “I felt [IvySCIP] did provide me with a lot of good information about my student, and how to identify the needs and then balance out . . . which ones were the most critical or which ones were not as critical,” (c) provides a comprehensive picture of students’ SEL strengths/needs—typical comments included, “It is so thorough. It goes through so many facets of the child and makes you look at them the way you wouldn’t or see things that aren’t right there on the surface, or obvious.” This category also included the one following minor theme: supports goal development for new/more challenging students—typical comments included, “[IvySCIP] would be . . . helpful for a brand new student,” and “I could see . . . using it for students who are more challenging for writing goals.”

The second category was high-quality goal builder and goals, which included the following three major themes: (a) positive response to goal builder—typical comments included, “I really liked that they generated goals, and you didn’t have to come up with them yourself,” and “IvySCIP goal builder . . . choices were exactly what I’ve been trying to write but not doing. OMG, it was so great!,” (b) creates measurable, high-quality goals containing all necessary components—typical comments included, “It was nice to make sure I had all the parts of the goal, to make sure it was measurable, and you had givens and settings. It really made sure you had each part of the goal you needed,” (c) provides a customizable SEL goal bank—typical comments included, “Goals can be super personalized, so you don’t feel at all limited.”

The third category was a comprehensive system linking goals to other resources, which included the following three major themes: (a) provides a comprehensive system for developing and implementing goals—typical comments included, “It’s not just an assessment—you can track progress over time, [select] lessons, do data collection, it has everything, so many different things to it. This is something districts have been looking for and it has everything in one snapshot,” (b) links goals to appropriate curricular materials—typical comments included, “I really liked being able to look for specific resources that . . . go along with the goal. I really loved that,” (c) links goals to progress monitoring resources—typical comments included, “One of the difficulties is . . . collecting data, and I feel [IvySCIP] helps me gather good information and gives me good direction.”

The fourth category was benefits teachers, which included the following four major themes: (a) easy to use and saves time—typical comments included, “IvySCIP is really informative and easy to use. I’m not very technologically savvy, and I was able to work with it without any difficulties or frustrations at all,” (b) increases teacher confidence—typical comments included, “IvySCIP helped show me that what I was doing was effective, because areas identified as deficit areas were where I was already providing IEP goals. The fact that IvySCIP was able to pick up on that was very exciting,” (c) provides information to support/share with colleagues—typical comments included, “The IvySCIP program would be a more streamlined form of communication between teachers,” (d) supports teachers new to this population and unfamiliar with crafting SEL goals—typical comments included, “I think it could especially help brand new people who really need to understand these students—I could almost be used as an educational tool.” This category also included one minor theme: provides information to support/share with parent—typical comments included, “I felt if I’d walked into an IEP meeting [with IvySCIP], I would have come in with so much data and information I wouldn’t have had previously.”

The fifth category was benefits students, which included the following major theme: useful with a broad range of students—typical comments included, “I think this could be used with any kiddos working on SEL.” This category also included the following minor theme: provides necessary resources for ASw/oID—typical comments included, “I would say for the higher functioning students there aren’t as many resources, so it is helpful.”

Social Validity

As part of an end-of-intervention product evaluation, participating instructors expressed their confidence in the value of IvySCIP for developing/tracking student goals. Using a Likert-type scale where 4 = “agree” and 5 = “strongly agree,” instructors gave both of the following statements an average score of 4.47: “IvySCIP software will be an effective way for special educators to develop appropriate SEL IEP goals for students with HF-ASD (high functioning autism spectrum disorders)” and “IvySCIP software will be an effective way for special educators to track progress toward meeting SEL IEP goals for students with HF-ASD.” In terms of more general impact of the tool, instructors gave an average score of 4.29 to the statement, “I believe IvySCIP software will improve social skills outcomes for children with HF-ASD.”

Discussion

Findings from this study validate extant research regarding the barriers instructors experience to create high-quality SEL IEP goals for their ASw/oID, including a lack of data-driven resources to help prioritize areas of students’ greatest need (Lucas et al., 2014), the need for training on how to write quality IEP goals (Russo-Campisi, 2020), and the importance of goal banks to ensure that the four key components of goals are always included (More & Hart-Barnett, 2014). Findings further provide preliminary evidence suggesting that the IvySCIP goal builder—which helps generate goals based on a comprehensive assessment of each student’s unique profile of strengths and needs—offers a promising tool for addressing these barriers and supporting instructors to develop high-quality SEL IEP goals. The following discussion focuses on key implications of this study for classroom practice, as well as summarizing limitations and proposing future directions.

One of the most important results of this study was the finding that using the IvySCIP goal builder resulted in higher quality SEL IEP goals. Our examination of pre-intervention IEP goals confirmed earlier findings indicating that IEP goals frequently lacked criteria for measurement (e.g., Hott et al., 2019; Ruble et al., 2010; and Sanches-Ferreira et al., 2013), without which a goal cannot be meaningfully tracked. The IvySCIP goal builder reminded instructors to include clear measurement criteria, as well as settings, givens, and specific target behaviors.

Related to this, goal development driven by IvySCIP assessment data appeared to result in instructors creating SEL goals that were more broadly representative of all five SEL domains. Specifically, instructors no longer focused primarily on social interaction, and to a lesser degree executive skills and critical thinking, and instead generated a corpus of goals that represented all five IvySCIP domains including emotion regulation, self-awareness and advocacy, and in a few cases, self-care. This broadening of goal focus may have to do with the fact that the use of data to support goal development (such as that provided via the IvySCIP strengths and needs assessment) (a) helps expand instructors’ conceptualization of SEL and (b) was therefore more likely to result in instructors focusing on areas of SEL need they would not have otherwise considered. Of note, all post-IvySCIP SEL goals focused on skill development that aligned with CASEL, as opposed to focusing on behavior reduction (as some of the pre-intervention IEP goals did).

Because IvySCIP goals are fully customizable, it was still possible for instructors to ignore the tool’s recommendations and create less complete goals. This occurrence was rare, however, with most participating instructors opting to retain all four goal components. While we were disappointed to find that more than 50% of IEP goals lacked measurement criteria prior to introduction of IvySCIP, we were excited to discover that 97% of goals included measurement criteria following use of the goal builder. The IvySCIP goal builder appears to offer a simple way of supporting creation of high-quality IEP goals while still granting instructors the freedom to customize goals as needed.

Another important finding had to do with instructors’ desire for data-driven goal selection and progress monitoring options. Almost all the instructors who participated in end-of-year interviews noted that this was one of the IvySCIP features they appreciated most. They liked the way the SEL strengths and needs assessment automatically fed into priority areas for goal development, and the fact that the IEP goal builder not only included measurement criteria but also provided options for tracking progress toward goal mastery. This finding indicates that while instructors may not know how to do it independently, they recognize the importance of data-driven instruction. Several reported they were aware that anecdotal evidence was inadequate for decision-making purposes, and that they had hoped for something like IvySCIP—a “one stop shop” in the words of a participating instructor—to help them feel more deliberate, informed, and confident when setting SEL goals for their students and monitoring their students’ progress.

As Rotter (2014) notes, the issue of including progress monitoring criteria is also an essential means by which schools demonstrate provision of a FAPE. In this way, IvySCIP and tools like it may also offer a means of supporting schools and districts to comply with IDEA. By prompting instructors to select settings, givens, specific target behaviors, and measurement criteria, goals are more likely to be written in a consistent manner. While some participating instructors came into our study already knowing how to create quality goals, use of IvySCIP helped elevate the quality of those instructors who entered the study with less skill in this area, and after introducing IvySCIP, goals were much more similar in structure.

A number of instructors noted they were easily able to integrate use of IvySCIP with other SEL resources they were using. IvySCIP includes a database of evidence-based practices (EBPs) drawn from the Frank Porter Graham Child Development Institute’s National Clearinghouse on Autism Evidence and Practice (Steinbrenner et al., 2020), and authors recommend the use of these practices in conjunction with IvySCIP assessment, IEP goal development, and data tracking.

Findings from this study should be interpreted with the following cautions in mind. First, our sample size of 40 was relatively small and homogenous. Most participating instructors were White women with significant prior experience in the classroom, which resulted in most feeling quite confident in their abilities to teach SEL and write SEL IEP goals for their ASw/oID prior to introduction of IvySCIP. A number of participants mentioned that IvySCIP would be of particular benefit to new instructors, and future research should include a larger and more diverse group—or even focus on IvySCIP as a means of supporting beginning teachers/related service providers and/or those who are relatively new to the field.

Second, end-of-year interview protocols did not focus specifically on the topic of IEP goals, and so the only information we were able to glean from transcripts was incidental. Fortunately, because the need for systems to support high-quality IEP goal development and progress monitoring is so great, this topic emerged spontaneously and repeatedly during almost every interview—resulting in a great deal of rich content. A more focused set of interview questions, however, might have helped elicit more nuanced information on both the state of the field and the perceived contribution of IvySCIP.

Additional limitations are as follows: (a) although IvySCIP goals remain fully customizable, the fact that the software prompts users to include settings, givens, and measurement criteria naturally increases the likelihood that goals created using the goal builder will include these components; (b) we were unable to provide detailed information on students’ social, emotional, academic, and behavioral functioning other than inclusion criteria and demographic information; (c) information was unavailable as to how goals created prior to IvySCIP were developed (although instructors frequently noted that formal assessment was not part of this process); and (d) social validity questions may have skewed toward reporting of favorable experiences because no direct questions were asked about perceived limitations of the tool.

Finally, though this study offers a starting point for examining issues related to SEL IEP goals for ASw/oID, the dearth of information currently available on this topic indicates that more research is needed to better serve this population. A significant body of research on IEP goals was published in the years after IDEA first became law. Since that time, a smattering of publications on this topic have emerged, but a comprehensive study is called for that (a) looks specifically at the needs of instructors writing SEL IEP goals for their autistic students and (b) tests tools such as IvySCIP more fully in terms of their ability to improve both goal quality and student outcomes.

Concluding Remarks

Overall, the findings from this study help delineate the types of challenges instructors currently face when developing SEL IEP goals for their K-5 ASw/oID, as well as introducing an innovative tool that appears to offer a means of effectively addressing these challenges. Given the improved IEP goal quality resulting from use of the IvySCIP goal builder, as well as the observed shift in content focus following introduction of a data-driven goal development process, we believe further research and development is warranted to ensure that tools such as IvySCIP can be made available to the instructors who would benefit most—particularly new teachers, or teachers who are less experienced in providing SEL instruction and/or working with ASw/oID. Given that the most recent research specifically addressing quality of IEP goals for this population was written more than 10 years ago (i.e., Ruble et al., 2010), this article provides an important update.

Footnotes

Authors’ Note

Some individuals prefer person-first language (e.g., “students with autism”), whereas others prefer identity-first language (e.g., “autistic students”). We used identity-first language throughout this article because that is what most of our students use. We also used the phrase “without intellectual disability” to describe our participants, some of whom have past diagnoses of either Asperger syndrome or high functioning autism. Characterizing this subgroup can be tricky, however, because intellectual ability does not necessarily correlate with degree of support needed, and even though our autistic participants did not experience concomitant intellectual disability, all experienced significant social cognition challenges.

Declaration of Conflicting Interests

Three authors of manuscript were also authors of IvySCIP. We do not stand to benefit financially, however.

Funding

The author(s) received financial support from the U.S. Department of Education, Institute of Education Sciences.

References

American Psychiatric Association. (2013). Diagnostic and statistical manual of mental disorders (DSM-5). American Psychiatric Publications.

Boavida

Aguiar

McWilliam

(2014). A training program to improve IFSP/IEP goals and objectives through the routines-based interview. Topics in Early Childhood Special Education, 33(4), 200–211. https://doi.org/10.1177/0271121413494416

Brackett

M. A.

Reyes

M. R.

Rivers

S. E.

Elbertson

N. A.

Salovey

(2012). Assessing teachers’ beliefs about social and emotional learning. Journal of Psychoeducational Assessment, 30(3), 219–236. https://doi.org/10.1177/0734282911424879

Brantlinger

Jimenez

Klingner

Pugach

Richardson

(2005). Qualitative studies in special education. Exceptional Children, 71(2), 195–207. https://doi.org/10.1177/001440290507100205

Bridgeland

Bruce

Hariharan

(2013). The missing piece: A national teacher survey on how social and emotional learning can empower children and transform schools. A Report for CASEL. Civic Enterprises.

Busby

Ingram

Bowron

Oliver

Lyons

(2012). Teaching elementary children with autism: Addressing teacher challenges and preparation needs. Rural Educator, 33(2), 27–35. https://doi.org/10.35608/ruraled.v33i2.416

Cappadocia

M. C.

Weiss

J. A.

(2011). Review of social skills training groups for youth with Asperger syndrome and high functioning autism. Research in Autism Spectrum Disorders, 5(1), 70–78.

Corcoran

R. P.

Cheung

A. C.

Kim

Xie

(2018). Effective universal school-based social and emotional learning programs for improving academic achievement: A systematic review and meta-analysis of 50 years of research. Educational Research Review, 25, 56–72.

Diehm

(2017). Writing measurable and academically relevant IEP goals with 80% accuracy over three consecutive trials. Perspectives of the ASHA Special Interest Groups, 2(16), 34–44. https://doi.org/10.1044/persp2.SIG16.34

10.

Durlak

J. A.

Weissberg

R. P.

Dymnicki

A. B.

Taylor

R. D.

Schellinger

K. B.

(2011). The impact of enhancing students’ social and emotional learning: A meta-analysis of school-based universal interventions. Child Development, 82(1), 405–432.

11.

Espelage

D. L.

Rose

C. A.

Polanin

J. R.

(2016). Social-emotional learning program to promote prosocial and academic skills among middle school students with disabilities. Remedial and Special Education, 37(6), 323–332.

12.

Findley

Ruble

McGrew

(2022). Individualized education program quality for transition aged students with autism. Research in Autism Spectrum Disorders, 91, 101900. https://doi.org/10.1016/j.rasd.2021.101900

13.

Gelbar

N. W.

Bruder

M. B.

DeBiase

Molteni

J. D.

(2018). A retrospective chart review of children with ASD’s individual education plans compared to subsequent independent psychological evaluations. Journal of Autism and Developmental Disorders, 48(11), 3808–3815.

14.

Hott

Jones

Rodriguez

Brigham

Martin

Mirafuentes

(2019). Are rural students receiving FAPE? A descriptive review of IEPs for students with social, emotional, or behavioral needs. Behavior Modification, 45(1), 13–38. https://doi.org/10.1177/0145445518825107

15.

Lucas

Gillaspy

Peters

Hurth

(2014). Enhancing recognition of high quality, functional IEP goals: A training activity for early childhood special education staff. Early Childhood Technical Assistance Center.

16.

Maenner

M. J.

Shaw

K. A.

Baio

Washington

Patrick

DiRienzo

Christensen

D. L.

Wiggins

L. D.

Pettygrove

Andrews

J. G.

Lopez

Hudson

Baroud

Schwenk

White

Rosenberg

C. R.

Lee

Harrington

R. A.

. . . Dietz

P. M.

(2020). Prevalence of autism spectrum disorder among children aged 8 years—Autism and developmental disabilities monitoring network, 11 sites, United States, 2016. Morbidity and Mortality Weekly Report Surveillance Summaries, 69(4), 1–12. https://doi.org/10.15585/mmwr.ss6904a1

17.

Miles

Huberman

(1994). Qualitative data analysis. (2nd ed.). Sage.

18.

Hart

(2013). Maximizing the use of electronic individualized education program software. TEACHING Exceptional Children, 45(6), 24–29. https://doi.org/10.1177/004005991304500603

19.

Hart-Barnett

(2014). Developing individualized IEP goals in the age of technology: Quality challenges and solutions. Preventing School Failure, 58(2), 103–119. https://doi.org/10.1080/1045988X.2013.782533

20.

Müller

Wood

Childress

Cannon

Dardick

(2022). Preliminary investigation of the relationship of Ivymount Social Cognition Package (IvySCIP) to similar assessment tools and impact on students with high functioning autism. Journal of International Special Needs Education. Advance online publication. https://doi.org/10.9782/JISNE-D-21-00011

21.

Rotter

(2014). IEP use by general and special education teachers. SAGE Open. Advance online publication. https://doi.org/10.1177/2158244014530410

22.

Rowland

Quinn

Steiner

(2015). Beyond legal: Crafting high-quality IEPs for children with complex communication needs. Communication Disorders Quarterly, 37(1), 53–62. https://doi.org/10.1177/1525740114551632

23.

Ruble

McGrew

(2013). Teacher and child predictors of achieving IEP goals of children with autism. Journal of Autism and Developmental Disorders, 43, 2748–2763. https://doi.org/10.1007/s10803-013-1884-x

24.

Ruble

McGrew

Dalrymple

Jung

(2010). Examining the quality of IEPs for young children with autism. Journal of Autism and Developmental Disorders, 40, 1459–1470. https://doi.org/10.1007/s10803-010-1003-1

25.

Russo-Campisi

(2020). The effects of eLearning instruction on the quality of written IEP goals and objectives. Temple University. https://doi.org/10.34944/dspace/3487

26.

Sanches-Ferreira

Lopes-dos-Santos

Alves

Santos

Silveira-Maia

(2013). How individualised are the individualised education programmes (IEPs): An analysis of the contents and quality of the IEPs goals. European Journal of Special Needs Education, 28(4), 507–520.

27.

Steinbrenner

Hume

Odom

Morin

Nowell

Tomaszewski

Szendrey

McIntyre

Yücesoy-Özkan

Savage

(2020). Evidence-based practices for children, youth, and young adults with autism. FPG Child Development Institute.

28.

Yell

Katsiyannis

Ennis

Losinski

Christle

(2016). Avoiding substantive errors in individualized education program development. TEACHING Exceptional Children, 49(1), 31–40. https://doi.org/10.1177/0040059916662204