Observations of Academic Instruction for Students With Significant Intellectual Disability

Abstract

As part of the validation process for alternate assessments, 39 classroom observations were conducted to gather data about current practices in providing academic instruction to students with significant intellectual disability. Using a standardized protocol, data were gathered using direct instructional observation, an Individualized Education Program review, and an analysis of student work. We found that teachers lacked age-appropriate materials, and most lessons and work samples required routine and recall responses. When teachers incorporated strategic or extended thinking activities into lessons, students were able to demonstrate their knowledge. The findings are suggested as a snapshot of current practices.

Keywords

alternate assessments academic instruction in special education

As a result of federal legislation, children with significant intellectual disability are expected to have access to and demonstrate progress in the general education curriculum (Individuals With Disabilities Education Act [IDEA], 1997; Individuals With Disabilities Education Improvement Act [IDEIA], 2004; No Child Left Behind [NCLB], 2002). To measure their progress, these students are now included in large-scale accountability systems. Specifically, those students who cannot take a regular assessment, even with appropriate accommodations, may take an alternate assessment based on alternate achievement standards (AA-AAS). Although there have been many challenges with implementing alternate assessments for students with significant intellectual disability, the mandate has dramatically influenced classroom practices in special education and called for scrutiny of the academic instructional methods of special education teachers, the performance of their students, and the validity of alternate assessment programs.

Validity evidence based on the consequences of testing fulfills a fundamental purpose of the validation process (American Educational Research Association, 1999)—to examine whether the benefits of changes to special education as a result of the assessment program are being realized. Therefore, investigating the impact of AA-AAS assessment programs on classroom practices is part of the validation process. The importance of the consequences of testing in relation to assessment programs in general and AA-AAS in particular has been articulated in the literature (Lane & Stone, 2002; Linn, Baker, & Dunbar, 1991; Marion & Pellegrino, 2006; Perie & Marion, 2008; Towles-Reeves, Garrett, Burdette, & Burdge, 2006).

In the first review of literature surrounding AA-AAS, Browder et al. (2003) noted that the promise of AA-AAS programs would include increased academic expectations for students with disabilities, increased access to the general curriculum and state academic content standards, and improvement of instructional decision making at the teacher and classroom level. The second review of the literature (Towles-Reeves, Kleinert, & Muhomba, 2009) confirmed that these themes remained relevant and added that future research should document the extent that AA-AAS programs are providing students with significant intellectual disability access to the general education curriculum. In recent years, a number of studies have investigated the impact of AA-AAS programs on academic instruction to students with significant intellectual disability. Most of the studies have utilized survey data (Cameto et al., 2010; Flowers, Ahlgrim-Delzell, Browder, & Spooner, 2005; Roach, Elliott, & Berndt, 2007; Towles-Reeves et al., 2006; Towles-Reeves & Kleinert, 2006; Young-Gyoung et al., 2006). However, none included direct observation of classroom instruction. Therefore, there is little observational data against which states can document the improved access to the general education curriculum as recommended by Towles-Reeves, Kleinert, and Muhomba (2009).

Two studies investigating relevant classroom practices through direct observation of students with significant intellectual disability in instructional settings compared student engagement in academic activities between inclusive and segregated settings. Wehmeyer, Lattin, Lapp-Rincker, and Agran (2003) observed 33 students with intellectual disability to ascertain the degree to which they were engaged in activities related to standards-based learning. Their findings suggested students in inclusive settings were more likely to be engaged in standards-based learning activities than when they were in a segregated setting. Similar findings regarding inclusion and standards-based learning activities were reported by Soukup, Wehmeyer, Bashinski, and Bovaird (2007). These studies provided information with respect to the physical settings in which students received standards-based instruction. More information is needed, however, to understand the quality and content of current instructional practices.

This study was undertaken by a consortium of three states working in collaboration with the University of Minnesota’s North Central Regional Resource Center (NCRRC). This consortium proposed to study the consequences of their respective AA-AAS systems using a model suggested by Lane and Stone (2002). This model includes the examination of classroom artifacts to strengthen the validity argument for large-scale assessment.

Because few studies have utilized direct classroom observations, little is known about current instructional practices for providing students with the most significant intellectual disability academic content aligned with the general education curriculum. The purpose of this study was to provide validity evidence associated with the AA-AAS in the consortium states and insight into classroom practices in special education specifically targeted to teach academic skills to students with significant intellectual disability. The results of this study provide a snapshot of student work, instructional practices, and Individualized Education Program (IEP) content measured against 25 indicators of quality. These observations have the potential to serve as insight to instructional practices against which states may measure and compare progress toward greater access to and progress in the general education curriculum for students with significant intellectual disability.

Method

Participants

There were two groups of participants in this study: students and teachers. All students who were observed qualified to take the AA-AAS. To take the AA-AAS, students must be unable to demonstrate their academic progress on the regular assessment, even with appropriate accommodations (IDEA, 1997). In addition, students’ IEP teams must decide that the AA-AAS is the appropriate assessment for the students to demonstrate what they know and can do. Although all students who are administered the AA-AAS meet these criteria, they are recognized to be a widely divergent group of individuals with extensively differing instructional needs (Kearns, Towles-Reeves, Kleinert, & Kleinert, 2009; Towles-Reeves, Kearns, Kleinert, & Kleinert, 2009; Towles-Reeves, Kleinert, et al., 2009).

Student characteristics are reported in Table 1. These data were collected through teacher interviews conducted by observers on the day of classroom observation. Observations were conducted in 18 elementary schools, 13 middle schools, 5 high schools, and 3 center-based/specialized schools. Most students did not require augmentative and alternative communication (AAC). Of those who did use AAC, most communicated using one symbol at a time. The majority of students were observed in a segregated setting. Three students were observed in specialized schools. Twenty-five percent of students were in general education for at least 40% of their day, and none of the observations occurred in inclusive classrooms. It should be noted that the students who were observed as part of this study were selected by their teachers and, therefore, cannot be assumed to be representative of the population.

Table 1.

Characteristics of Students Observed.

Student characteristic	Percentage of students
Expressive communication
Uses symbolic language	84
Uses intentional communication	8
No clear use of systematic communication	8
Receptive communication
Independently follows 1–2 step directions	55
Requires additional cues	39
Alerts to sensory input	3
Uncertain response to sensory stimuli	3
Vision
Within normal limits	79
Corrected vision within normal limits	15
Low vision	0
No functional use of vision, unable to determine	3
Hearing
Within normal limits	90
Corrected hearing loss within normal limits	4
Hearing loss aided, but still with a significant loss	3
Profound loss, even with aids	3
Motor
No significant motor dysfunction	84
Requires adaptations to support motor functioning	11
Uses wheelchair, or other equipment for most activities	0
Needs personal assistance for most/all motor activities	5
Engagement
Initiates and sustains social interactions	55
Responds with social interaction, but does not initiate or sustain	37
Alerts to others	8
Does not alert to others	0
Reading ability
Reads fluently with critical understanding	3
Reads fluently with basic understanding	41
Reads basic sight words	39
Aware of print or Braille	5
No observable awareness of print or Braille	13
Mathematics ability
Applies computational procedures to solve real-life problems in a variety of contexts	15
Does computational procedures with or without a calculator	54
Counts with 1:1 correspondence	15
Counts to 5 by rote	5
No observable awareness of numbers/numeracy	10

Teachers were identified as potential participants for this study based on their completion of a web-based survey focused on teacher perceptions and practices associated with the state AA-AAS system (Restorff, Sharpe, Abery, Rodriguez, & Kim, 2012). This survey was administered to a stratified random sample of schools in each of the consortium states. There were 401 completed survey responses from the three states.

The stratification plan used for the surveys was not acceptable for this study due to a number of factors. Due to the small number of students in some districts who take the AA-AAS, teacher and student anonymity could not be guaranteed. Furthermore, the survey respondents did not adequately represent the wide geographic areas of the collaborating states. An alternative stratification plan was therefore designed for selecting potential schools for observation.

Using the U.S. Bureau of Census’s (2010) definition of urban areas, survey respondents were stratified into two groups: urban or non-urban, based on the location of their schools. The Bureau defines “urban” as areas that have a population density of at least 1,000 people per square mile and surrounding blocks that have an overall density of at least 500 people per square mile (http://www.census.go/geo/www/us/ua_2k.html). A random sample of 45 respondents was drawn from each state with proportional representation. That is, if according to the U.S. Census, 25% of a state’s students were classified as attending districts in urban areas, 25% of the sample was drawn from respondents from urban districts.

In two of the three states, this stratification yielded proportional representation; however, in the third state, the stratification resulted in an overrepresentation of teachers in one large urban school district. For this state, intermediate units across the state were collapsed from 13 strata to 4 strata, and 45 schools were randomly drawn from that list. All of the teachers who were observed held special education licensure and taught at least one student who took the AA-AAS during the school year in which the observations took place.

Instrumentation

The study used an observation protocol developed by the National Alternate Assessment Center (NAAC; 2010) specifically intended to help states collect evidence with respect to the consequential validity of their alternate assessment systems. With permission from the authors, the observation protocol was slightly modified for this particular study. The NAAC protocol organizes an analysis of academic content delivered to students with significant intellectual disability around three components: an observation of a teaching session focused on academic content, a review of the student’s IEP, and an analysis of a sample of the student’s work. In addition, the protocol contains a teacher interview and gathers student profile data. The protocol and online training for its use are available through the NAAC website upon request (www.naacpartners.org).

The NAAC protocol was developed and validated through a review of the recent literature surrounding effective practices for students with significant intellectual disability and classroom walk-through tools for general educators. The organization of the student profile was drawn from the Learner Characteristics Inventory (LCI; Kearns, Kleinert, Kleinert, & Towles-Reeves, 2006; Towles-Reeves, Kearns, et al., 2009). The IEP analysis was drawn from the work of Hunt, Goetz, and Anderson (1986). A draft of the instrument was sent to nationally recognized experts in the field of education for students with significant intellectual disability and was revised based on expert feedback. The revised protocol was piloted in Kentucky, the Territory of Guam, and Yap State, Federated States of Micronesia in elementary, middle, and high school classrooms. Interrater agreement on the revised instrument reached 97% (Spencer et al., 2012).

The activity observation component of the protocol consists of 20 indicators completed by observers following a classroom observation of a minimum of 20 min or until the specific instructional task/activity under observation has been completed. Indicators 1 through 5 and Indicator 20 are multiple-choice/descriptive items regarding the instructional setting, activity type, group arrangements, instructional delivery, instructional level, and level of independence of student responses. Indicators 6 through 19 are a checklist in which observers record a plus if they observe the indicator and a minus if they do not. These indicators focus on aspects of the instructional process, including whether instruction is chronologically age/grade-level appropriate, is offered in an inclusive setting and using age-appropriate materials, is accessible and allows for appropriate options to demonstrate understanding, requires the student to respond at more than one level of depth of knowledge, and includes the number or percent of correct responses and level of prompting provided. There is a space for notes next to each indicator.

The IEP component requires the observer to list the number of objectives or benchmarks on the student’s IEP by content area. The first six IEP benchmarks are evaluated by the observer to determine whether the benchmark is linked to a content standard (either general or extended). Due to time constraints in this study, observers were directed to evaluate linkage for only the first six benchmarks on the students’ IEPs. Because student IEPs could not be removed from the school, observers were required to carry a copy of the extended standards with them to the observation and conduct the analysis on site. The IEP component of the protocol also collects data on student accommodations and supports, the number of minutes specified for the student to receive instruction in the general education setting, a description of supplementary aides and services, and the need for AAC. Finally, the observer notes the students’ IEPs, specifies the need for an alternate assessment, and lists the justification.

The work sample component consists of 10 items. Items 1 through 9 are in checklist format, requiring the observer to record plusses if they observe the indicator. The checklist includes items on age-appropriate content, alignment of the work to standards, depth of knowledge, tailoring, and the extent of teacher feedback (such as a grade, percentage correct, or indication of level of prompting). The last item is a multiple-choice item where the observer compares the work sample to chronologically age-appropriate, grade-level materials.

The final component of the protocol is a teacher interview. The teacher interview contains questions about the student’s behavior, attention, and responding during the activity observation compared with the student’s behavior, attention, and responding on a typical day. The interview is completed immediately following the observation.

Procedures

State Department of Education personnel composed an introductory email informing the teachers and their principals that they had been randomly selected to participate in this part of the study. Teachers at the respective schools were then contacted directly by observers. If the teachers agreed to participate, a time was established for the school visit. Teachers who took part in this study received a US$100 gift certificate to a special education supply vendor that could be used to purchase materials for their classrooms. After contacting schools, 39 teachers agreed to be in the study (State 1 = 14, State 2 = 10, State 3 = 15).

Observers were selected by their respective state consortium representatives. The state consortium representatives had reviewed the NAAC protocol and were well aware of the prerequisite skills required to conduct the observations, such as the ability to accurately assess age-appropriate curricular materials and professional training in special education. Due to the highly sensitive nature of direct observation of individuals with disability in classroom settings, the observers selected by the state representatives all held positions that allowed them access to student records and classrooms (i.e., all individuals were employed by the state).

In one state, observers were drawn from intermediate units. These individuals are employed specifically to provide training and resources to special educators within their member districts and are considered experts. In another state, observers were selected from state education consultants. These consultants included former directors of special education, former school superintendents, and special education teachers working in higher education. In the third state, Department of Education employees who provide leadership and support to special educators in the field were trained to conduct classroom observations. Although the group of observers from each state was drawn from a wide range of special education licensure and experience, all observers held the minimum of a master’s degree in an educational field of study. Their years of experience ranged from 9 to 34 years, with an average of 14 years. In addition, the state representatives who recruited and selected these observers were deeply invested in the results of this study and were highly selective in their choice of observers.

To achieve a high level of fidelity in the administration of the NAAC protocol, NCRRC project staff conducted a full day of training with observers within each state. After background information that focused on the overall responsibilities of observers was provided, observers were presented with detailed instruction with respect to the completion of each part of the NAAC protocol. Operational definitions were reviewed for behaviors to be observed. During training, each observer completed the NAAC protocol using a videotape of an instructional session, a sample IEP, and a sample of student work. Under the supervision of trainers, observers then independently scored a variety of samples and their analyses were compared with expert scoring. Training with respect to each component of the NAAC protocol continued until adequate interrater reliability (>80%) was achieved.

Although the NAAC protocol has three distinct components, the protocol contains a number of themes. One such theme is instruction on age-appropriate, grade-level specific general curriculum. This theme is sampled multiple times across the components. For observers to make a valid judgment, observers used the state’s alternate achievement standards to examine IEP goals and objectives during training. In the field, observers were required to carry a copy of the alternate standards with them to the site to evaluate the degree to which the IEP, lesson activity, and work sample aligned with the grade-level standards. An example of the training provided to observers is available on the NAAC website (www.naacpartners.org).

After training, observers contacted teachers and provided an overview of the observation. Teachers were informed observers would spend 25 to 45 min observing an academic lesson (language arts, mathematics, or science) delivered to a student who took the AA-AAS. They were also informed that the observers would review the student’s IEP, examine a work sample of 3 to 5 items generated by that student, and conduct a teacher interview about the lesson just observed. Participating teachers were allowed to select the student, the lesson, and specify the time and date of the observation. Ideally, further interrater reliability checks would have been conducted during observations; however, given the sensitivity of the observation and data, the level of credentials required for legal student observation and record review, and the dispersion of observation sites throughout large geographic areas, this was not feasible.

Observers redacted personal identifying information from the protocols prior to forwarding them to state Department of Education staff. To further ensure student privacy, protocols were mailed in preaddressed UPS or FedEx envelopes directly to respective departments of education. State representatives conducted a final review of the protocols to ensure privacy was maintained and forwarded the protocols to NCRRC for analysis.

Data Analysis

Descriptive statistics (frequencies and proportions) were calculated for each section of the observation protocol.

Results

The observation protocol was coded and the results were aggregated across states. Results are presented by sections of the protocol. Care has been taken to limit the results reported here to those providing an accurate picture of the data. For example, while data were collected indicating “age-appropriate peers” were not present during the instruction, the nature of the observation was such that most were conducted with one student or a small group in a resource room. Thus, while data were collected indicating that very few lessons were provided in the presence of general education peers, results should not be interpreted to mean that these students had no inclusion opportunities. All data reported in the results section were collected from the protocols.

Activity Observation

Of the 39 classrooms in which lessons were observed, 14 lessons were coded as academic (language arts, math, or science) and chronologically age appropriate. In all, 20 were coded as academic, but not chronologically age appropriate, and 5 were coded as life skills or other. Most observations were conducted for language arts lessons (25), followed by mathematics (8), science (3), and other (3). The level of prompting for student responses during instruction was recorded with 24% of responses classified as independent and correct, 68% prompted and correct, 3% independent but incorrect, and 5% (two students) prompted but incorrect. Results of the observation indicators are shown in Table 2.

Table 2.

Activity Indicators for Classroom Observation.

Activity indicator	Observed %
Chronologically age-appropriate materials	36
The instruction allows appropriate option(s) for the student to demonstrate what they know	72
The instruction provides various appropriate options for the student to have access to the instruction on the grade level	62
The instruction provides various options to interest student and maintain his or her attention	77
Traditional “functional” skills embedded as needed within naturally occurring classroom routines	85
Instruction requires student to perform at more than one level of depth of knowledge	67
The student has a way to communicate that is appropriate for his/her communication level and includes the appropriate content	90
Student response demonstrating understanding of content is appropriately tailored and specifically designed	59
Student performs at more than one level of depth of knowledge	64
The student evaluates his or her own performance	51
The student demonstrates social behavior appropriate to the task and environment	90
The student maintains appropriate attention	95

There were 12 observation indicators observers marked as either present or absent during the observation of the lesson. In all, 4 were reported frequently, 5 were reported somewhat less frequently, and 3 were reported in less than 60% of the observations. The observations made most frequently regarded student behavior. More than 85% of the observations indicated that the target students displayed appropriate social behavior, maintained their attention during the lesson, and possessed a way to communicate that contained appropriate content. Furthermore, most observers reported that traditional functional skills were embedded into the lesson.

Observations made less frequently (62%–77%) were the requirement for the student to perform at more than one depth of knowledge and the students’ actual performance at more than one depth of knowledge. Also reported with this frequency was the provision of options for students to access the instruction on grade level, and that the teacher provided a way to interest the students and maintain the students’ attention. In terms of depth of knowledge, 90% of students were observed in lessons that required routine and recall. As the complexity of the lessons increased, the percentage of students performing at that level decreased. Only 12% to 15% of students were required to perform at the strategic or extended levels. It is notable that although only 67% of the lessons required students to perform at more than one depth of knowledge, 64% of students performed at more than one level. This suggests that when teachers provided instruction at more than one depth of knowledge, students were able to meet that challenge. Observation indicators noted least frequently were the use of chronologically age-appropriate materials (36%) and specific tailoring of the lesson to allow student response (59%). The opportunity for the student to evaluate his or her own performance was reported at 51%.

IEP Review

The purpose of the IEP review was to ascertain the number and type of objectives and benchmarks written into the students’ IEPs. IEP objectives were counted at their most discrete level (e.g., benchmarks) and classified as reading/language arts, mathematics, science, other academic, communication, social/behavioral, vocational, self-care, or other functional. The total number of objectives ranged from 3 to 35 with a mean of 15. Figure 1 represents the ranges for IEP objectives by subject.

Figure 1 indicates almost all IEPs contained objectives for reading, math, and functional skills. Only 3 of 39 IEPs contained science objectives. Of those not containing science objectives, the objectives comprised less than 25% of the total objectives included in the IEP.

Work Sample

Observers were directed to obtain and analyze three to five pieces of student work, including as a first priority, any work sample generated during the lesson observed. It should be noted that teachers were made aware that the observer would be asking for samples of the student work. Therefore, these samples provide a picture of “best foot forward.” Observers noted that almost all (92%) students observed were capable of producing a work sample. Of these students, 85% of the teachers supplied a student work sample for the observers to analyze.

Observation indicators for the student work protocol were applied to the total sample of work, as opposed to each specific segment. Observers found that 33% of the work samples were aligned with age-appropriate, grade-level content standards, and an additional 10% to the alternate standards. Of the remaining 57%, most observers noted the work sample was not aligned to age-appropriate, grade-level content standards. A substantial majority (69%) found the materials did not look substantively like materials being used by the students’ general education counterparts.

Depth of knowledge (Webb, 2007) was a consideration for the analysis of the work sample and the lesson. Observers looked for evidence the work sample, as designed, called for the student to perform at more than one level of depth of knowledge. In addition, they noted the specific depths of knowledge represented in the work sample and determined whether the work sample required the student to perform at least one activity at the application or generalization level. Seventy-three percent of the work samples reflected more than one level of depth of knowledge in their design and 32% of the work samples required performance at the application level. Table 3 summarizes the findings of depth of knowledge required by the work samples and the lessons, noting that some work samples and lessons required more than one. As shown in Table 3, almost all students were asked to engage in recall/routine instruction; a much smaller percentage of students were required to demonstrate a greater depth of knowledge.

Table 3.

Required Depth of Knowledge.

	Percentage of students
Depth of knowledge	Work sample	Lesson
Recall/routine	91	90
Discriminate similarities/differences	53	50
Application	32	44
Strategic	15	16
Extended	12	11

Work samples were examined for access and response accommodations. All observers reported that the samples had appropriate access accommodations where needed (e.g., “chunking” or providing additional white space). However, 27% did not reflect an appropriate response accommodation. Thirty-two percent of work samples were graded and 7% contained an indication of the number correct or the percentage correct. Only one work sample analyzed contained both an indication of an independent response and the level of prompting required.

Teacher Interviews

Teacher interviews consisted of two parts: an evaluation of the student’s performance during the observation and a survey of the types and frequency of professional development in which the teachers had participated. Immediately following each observation, teachers were asked to compare the student’s performance during the observation with his or her typical performance across four behavioral dimensions: performance, communication, attention, and level of independence. Most teachers reported that their students responded to the lesson in a typical fashion, with 95% stating the students’ performance was typical, 92% stating the students’ communication was typical, 77% stating the students’ attention was typical (15% reported students’ attention as atypically high), and 90% stating the students’ level of independence was typical. Teacher ratings of student responses under observation conditions therefore suggest the behavior observed was typical for the student in the instructional context.

In regard to professional development, Table 4 summarizes the type and frequency of the professional development activities provided to teachers. Most teachers reported receiving some type of professional development regarding providing access to the general education curriculum. Formal professional development and technical assistance occurred less frequently than school-based or teacher-to-teacher professional development. The highest frequencies of professional development were reported for school-based and teacher-to-teacher activities where 44% of the teachers stated that they engaged in these activities on a weekly basis. Forty-one percent of teachers reported receiving formal professional development on an annual basis.

Table 4.

Type and Frequency of Professional Development Activities Provided to Teachers for Access to the General Education Curriculum.

Frequency	Formal PD (%)	School-based (%)	Teacher-to-teacher (%)	Technical assistance (%)
Percentage selecting	82	87	88	90
One time	5		3	8
Annually	41	5		28
Quarterly	23
Monthly	18	39	10	41
Weekly	3	44	44
Daily	3		28
As needed			3	5

Note. PD = professional development.

Discussion

Students

The students observed for this study clearly have more academic skills than their counterparts in other studies using comparable measurement characteristics (Kearns et al., 2006; Towles-Reeves, Kearns, et al., 2009), and their performance cannot be interpreted as representative of the 1% population. However, the observations made concerning their instruction, IEPs, and work samples may represent what those students who are most capable of learning the academic content are receiving in terms of instruction. These data are important because with regard to where the bar is set for student achievement, this is the top of the bar. It is unlikely students who have fewer academic skills are faring better than these students in terms of the academic instruction provided to them.

Activity Observation

A number of observations can be made regarding these data, and to some extent, they suggest what teachers are doing well and frequently, as well as point to areas where professional development and other supports are most needed. The absence of age-appropriate materials may illustrate a paucity of materials that bridge the gap between challenging grade-level content and a reduction in the complexity of the material that does not compromise age-appropriate presentation. The lack of tailoring student responses suggests that teachers might require more professional development in providing response accommodations for students.

Most students were required to perform at the recall and routine knowledge level, and only about one half of the students observed were required to evaluate their own performance. Considering a large number of students in this study are among the least disabled of the 1% population, there would appear to be a need to focus on providing professional development to teachers with respect to how they can engage their students in higher order learning tasks requiring application, strategic, and extended thinking.

Work Sample

The work samples reflected patterns similar to those found in the observation of instruction. More than half of the work samples were not age-appropriate in their appearance. Furthermore, the samples often did not indicate the percentage or number correct and only required students to perform at recall or routine levels. Response accommodations lagged behind access accommodations, and very few of the work samples indicated the level of prompting required or an indication of an independent response. These observations suggest teachers of students with significant intellectual disability need more clarity in regard to the expectations of their accountability for students’ work. It is not surprising that those practices associated with general education such as grading and self-evaluation are not part of the repertoire of many special education teachers who may previously have only been responsible for ensuring mastery of a mostly functional curriculum.

IEP

A wide range of variability was found in the composition of the IEPs of the students who took part in this study. The number of overall objectives/benchmarks on the 39 IEPs analyzed ranged from a low of 3 to a high of 35. Of the IEPs reviewed, all contained functional skills. In terms of academic content, all contained at least one reading objective, most contained mathematics objectives, but only 3 contained science objectives.

Because little is known about the composition of student IEPs by subject, these data may suggest a picture of transition from functional curricula to content-based instruction. As standards-based IEP objectives are written to support specific skills that students need to master academic content, the extent to which those skills required for science will differ from those that students need in mathematics and reading/language arts is emerging. As the science AA-AAS are developed and become operational, we might expect to see more science objectives written into IEPs in the future. Because there are no other studies that report these data, the proportion of objectives by subject areas presented might best be conceptualized as a snapshot in time, against which future progress could be measured.

Study Limitations

There were several limitations to the study. Although attempts were made to randomly select teachers from survey respondents, of the 130 teachers selected into the study, 39 agreed to participate. Students who participated in this study appeared to have fewer academic challenges than many of the students who take the AA-AAS. These data should, therefore, be interpreted as not fully representative.

Ideally, observers for this study would have been graduate students from the university trained in using the NAAC protocol for classroom observation. However, access to students with significant intellectual disability in a classroom setting is difficult. Numerous levels of consent and assent are required to view student records and gain access to the classroom. In addition, all of the states in the study are geographically large and would have required extensive travel to remote areas. Therefore, the use of observers selected and trained by the research team and employed by the university was not feasible. Instead, observers were drawn from current employees of the school systems of students in the sample. Both the research team and the state consortium representatives were confident that these individuals were highly capable of conducting the observations with fidelity; however, there were wide differences in their backgrounds and roles across and within the three states. Although observers were trained and interrater reliability was measured, reliability checks were not conducted in the field due to limitations of time and resources.

Analyzing IEP objectives for their alignment with content standards was time-consuming. Because IEPs were not copied or removed from the school buildings, that analysis had to be completed within the observation visit. Therefore, the analysis of IEP objectives and how they were linked to standards was limited to the first six objectives on the students’ IEPs.

One View

Studies reporting the practices of special education teachers with regard to academic instruction for students with significant intellectual disability are extremely limited. An evaluation of whether the field is progressing or standing still with respect to this area is therefore beyond the scope of this study. The availability of a standardized tool for measuring instructional practices (i.e., the NAAC observation protocol), however, will enable direct comparisons to be made in the future.

Periodic monitoring and similar studies to gauge the changes of classroom instructional practices will help determine the progress that educators are making toward providing access to the general education curriculum and increased student performance. Further study on the relationship of quality instruction as measured by the NAAC protocol and student performance on the AA-AAS as well as the relationship between professional development and the indicators of quality would be additional avenues to explore.

Footnotes

Acknowledgements

Special thanks to the following individuals, without whom this work would not have been possible: Michael Sharpe, Director, NCRRC; Sandra Berndt, Wisconsin DPI; Lynda Lupp, Pennsylvania Department of Education; Audrey Kappel, Pennsylvania Technical Assistance Network; Eva Kubinski, Wisconsin DPI; and Marilyn Roberts, Michigan Department of Education. Also thanks to Jacqueline Kearns and Michael Burdge of the National Alternate Assessment Center for the use of the NAAC Observation Protocol and training materials.

Authors’ Note

The opinions expressed do not necessarily reflect the position or policy of the Office of Special Education Programs, and no official endorsement should be inferred.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was supported by the U.S. Department of Education Office of Special Education Programs (Grant H373X070020).

References

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: Author.

Browder

D. M.

Spooner

Algozzine

Ahlgrim-Delzell

Flowers

Karvonen

(2003). What we know and need to know about alternate assessment. Exceptional Children, 70, 45–61.

Cameto

Bergland

Knokey

Nagle

K. M.

Sanford

Kalb

S. C.

Ortega

(2010). Teacher perspectives of school-level implementation of alternate assessments for students with significant cognitive disabilities: A report from the national study on alternate assessments (NCSER 20103007). Retrieved from http://ies.ed.gov/pubsearch/pubsinfo.asp?pubid=ncer20103007

Flowers

Ahlgrim-Delzell

Browder

Spooner

(2005). Teachers’ perceptions of alternate assessments. Research and Practice for Persons with Severe Disabilities, 30, 81–92. doi:10.2511/rspd.30.2.81

Hunt

Goetz

Anderson

(1986). The quality of IEP objectives associated with placement on integrated versus segregated school sites. Journal of the Association for Persons With Severe Handicaps, 11, 125–130.

Individuals with Disabilities Education Act (IDEA) of 1997, 20 U.S.C. §1400 et seq.

Individuals with Disabilities Education Improvement Act (IDEIA) of 2004, 20 U.S.C. § 1400 et seq.

Kearns

J. F.

Kleinert

H. L.

Kleinert

J. O.

Towles-Reeves

(2006). Learner Characteristics Inventory. Lexington, KY: University of Kentucky, National Alternate Assessment Center.

Kearns

J. F.

Towles-Reeves

Kleinert

H. L.

Kleinert

J. O.

(2009). Characteristics of and implications for students participating in alternate assessments based on alternate academic achievement standards. The Journal of Special Education, 45, 3–14. doi:10.1177/0022466909344223

10.

Lane

Stone

C. A.

(2002). Strategies for examining the consequences of assessment and accountability programs. Educational Measurement, 21, 23–30. doi:10.1111/j.1745-3992.2002.tb.00082.x

11.

Linn

R. L.

Baker

E. L.

Dunbar

S. B.

(1991). Complex, performance-based assessment: Expectations and validation criteria. Educational Researcher, 20, 15–21. doi:10.3102/0013189X020008015

12.

Marion

S. F.

Pellegrino

J. W.

(2006). A validity framework for evaluating the technical quality of alternate assessments. Educational Measurement: Issues and Practices, 25, 47–57. doi:10.1111/j.1745-3392.2006.00078.X

13.

National Alternate Assessment Center. (2010). Student/program observation tools. Retrieved from http://www.naacpartners.org/toolsforAA/obsTools.aspx

14.

No Child Left Behind Act of 2001, 20 U.S.C. §§ 6301 et seq. (2002)

15.

Restorff

D. E.

Sharpe

M. N.

Abery

B. H.

Rodriguez

M. C.

Kim

N. K.

(2012). Teacher perceptions of alternate assessments based on alternate achievement standards: Results from a three state survey. Research and Practice for Persons with Severe Disabilities, 37, 185–198.

16.

Roach

A. T.

Elliott

S. N.

Berndt

(2007). Teacher perceptions and the consequential validity of an alternate assessment for students with significant cognitive disabilities. Journal of Disability Policy Studies, 18, 168–175. doi:10.1177/10442073070180030501

17.

Soukup

Wehmeyer

M. L.

Bashinski

Bovaird

(2007). Classroom variables and access to the general curriculum for students with disabilities. Exceptional Children, 74, 101–120.

18.

Spencer

M. L.

Burdge

De Leon

Clayton

Quitugua

Guerrero

Macapinlac

(2012). Facilitating program improvement for students with significant cognitive disabilities: Baseline documentation of instructional programs in six pacific entities. Micronesian Educator, 16, 5–19.

19.

Towles-Reeves

Garrett

Burdette

P. J.

Burdge

(2006). Validation of large-scale alternate assessment systems and their influence on instruction: What are the consequences? Assessment for Effective Intervention, 31, 45–57. doi:10.1177/073724770603100304

20.

Towles-Reeves

Kearns

Kleinert

(2009). The learning characteristics of students taking alternate assessments based on alternate achievement standards. Journal of Special Education, 42, 241–254. doi:10.1177/0022466907313451

21.

Towles-Reeves

Kleinert

(2006). The impact of one state’s alternate assessment upon instruction and IEP development. Rural Special Education Quarterly, 25(3), 31–39.

22.

Towles-Reeves

Kleinert

Muhomba

(2009). Alternate assessments: Have we learned anything new? Exceptional Children, 75, 233–252.

23.

U.S. Bureau of Census. (2010). Census 2010 urban and rural classification. Retrieved from http://www.census.gov/geo/www/ua/ua_2k.html

24.

Webb

N. L.

(2007). Issues related to judging the alignment of curriculum standards and assessments. Applied Measurement in Education, 20, 7–25. doi:10.1080/08957340709336728

25.

Wehmeyer

M. L.

Lattin

Lapp-Rincker

Agran

(2003). Access to the general curriculum of middle-school students with mental retardation: An observational study. Remedial and Special Education, 24, 262–272. doi:10.1177/07419325030240050201

26.

Young-Gyoung

Angell

M. E.

O’Brian

M. O.

Strand

K. H.

Fulk

B. M.

Watts

E. H.

(2006). Relationships among teachers’ perspectives, self-reported practices, and concerns related to an alternate assessment system. Teacher Education and Special Education, 29, 83–97. doi:10.1177/088840640602900201