Corresponding with the 30th year following the passage of P.L. 94-142, a task force of the Council for Exceptional Children Division of Research published a set of articles outlining quality indicators (QIs) for research and evidence-based practice in special education. This article details the operationalization and application of QIs for single-participant designs in the area of academic instruction for students with emotional and behavioral disorders. The method used was a descriptive analysis of special education research (1975—2005). The outcomes reveal (a) more than half of the eligible studies failed to meet minimal QIs, (b) there were common variations across QIs, and (c) numerous interventions are supported by studies that met minimal QIs. Future research and limitations of the study are provided.
Access Center. (n.d.). Strategies to improve access to the general education curriculum. Retrieved from http://www.k8accesscenter.org/training_resources/documents/Research%20Supported%20Strategies%20Chart.doc
2.
Albers, A.E., & Greer, R.D. ( 1991). Is the three term contingency trial a predictor of effective instruction? Journal of Behavioral Education, 1, 337-354.
3.
American Association on Mental Retardation Board of Directors. (2003). Resolution of the AAMR on evidence-based research and intellectual disability. Retrieved from http://www.aaidd.org/media/PDFs/resolution_research.pdf
4.
American Psychiatric Association. (2000). Diagnostic and statistical manual of mental disorders (4th ed., text revision). Washington, DC: Author.
5.
Babyak, A.E., Koorland, M., & Mathes, P.G. ( 2000). The effects of story mapping instruction on the reading comprehension of students with behavioral disorders. Behavioral Disorders, 25, 239-258.
6.
Bell, K., Young, R., Blair, M., & Nelson, R. ( 1990). Facilitating mainstreaming of students with behavioral disorders using classwide peer tutoring. School Psychology Review, 19, 564-573.
7.
Borman, G.D. ( 2002). Experiments for educational evaluation and improvement . Peabody Journal of Education, 77(4), 7-27.
8.
Bradley, R., Henderson, K., & Monfore, D.A. ( 2004). A national perspective on children with emotional disorders . Behavioral Disorders, 29, 211-223.
9.
Brook, R.H., Chassin, M.R., Fink, A., Solomon, D.H., Kosecoff, J., & Park, R.E. ( 1986). A method for the detailed assessment of the appropriateness of medical technologies. International Journal of Technology Assessment in Health Care, 2, 53-63.
10.
Browder, D.M., Spooner, F., Ahlgrim-Delzell, L., Harris, A.A., & Wakeman, S. ( 2008). A meta-analysis on teaching mathematics to students with significant cognitive disabilities. Exceptional Children, 74, 407-432.
11.
Browder, D.M., Wakeman, S.Y., Spooner, F., Ahlgrim-Delzell, L., & Algozzine, B. ( 2006). Research on reading instruction for individuals with significant cognitive disabilities. Exceptional Children, 72, 392-408.
12.
Cade, T., & Gunter, P.L. ( 2002). Teaching students with severe emotional or behavioral disorders to use a musical mnemonic technique to solve basic division calculations . Behavioral Disorders, 27, 208-214.
13.
Caldwell, N.K., Wolery, M., Werts, M.G., & Caldwell, Y. ( 1996). Embedding instructive feedback into teacher-student interactions during independent seat work. Journal of Behavioral Education , 6, 459-480.
14.
Campbell, S.M., Braspenning, J., Hutchinson, A., & Marshall, M. ( 2002). Research methods used in developing and applying quality indicators in primary care. Quality and Safety in Health Care , 11, 358-364.
15.
Carr, S.C., & Punzo, R.P. ( 1993). The effects of self-monitoring of academic accuracy and productivity on the performance of students with behavioral disorders. Behavioral Disorders, 18, 241-250.
16.
Chard, D.J. ( 2004). Maintaining the relationship between science and special education. Journal of Learning Disabilities, 37, 213-217.
17.
Cook, B.G., & Tankersley, M. (2007). A preliminary examination to identify the presence of quality indicators in experimental research in special education. In J. B. Crockett, M. M. Gerber, & T. J. Landrum (Eds.), Achieving the radical reform of special education: Essays in honor of James M. Kauffman (pp. 189-212). Mahwah, NJ: Lawrence Erlbaum.
18.
Creswell, J.W. ( 1998). Qualitative inquiry and research design: Choosing among five traditions. Thousand Oaks, CA: Sage.
19.
Cybriwsky, C.A., & Schuster, J.W. (1990). Using constant time delay procedures to teach multiplication facts. Remedial and Special Education, 11, 54-59.
20.
Falk, K.B., & Wehby, J.H. ( 2001). The effects of peer-assisted learning strategies on the beginning reading skills of young children with emotional or behavioral disorders . Behavioral Disorders, 26, 344-359.
21.
Fan, X., & Chen, M. ( 2000). Published studies of interrater reliability often overestimate reliability: Computing the correct coefficient. Educational and Psychological Measurement, 60, 532-542.
22.
Finn, E. C., Rotherham, A. J., & Hokanson, C. R. (Eds.). (2001). Rethinking special education for a new century . Washington, DC: Thomas Fordham Foundation.
23.
Franca, V.M., Kerr, M.M., Reitz, A.L., & Lambert, D. ( 1990). Peer tutoring among behaviorally disordered students: Academic and social benefits to tutor and tutee. Education and Treatment of Children, 13, 109-129.
24.
Garland, C.W. ( 1995). Reactions from the field. DEC recommended practices: Who uses them? Journal of Early Intervention, 19, 18-20.
25.
Gersten, R., Fuchs, L., Compton, D., Coyne, M., Greenwood, C., & Innocenti, M. ( 2005). QI’s for group experimental and quasi-experimental research in special education. Exceptional Children, 71, 149-164.
26.
Graham, S. ( 2007). Evidence-based practice: Yes, but with cautions and recommendations . Retrieved from http://www.cecdr.org/news.cfm?id=FD48BC67-1422-201A-559D77EE-A2AEC4E0
27.
Grskovic, J.A., & Belfiore, P.J. (1996). Improving the spelling performance of students with disabilities. Journal of Behavioral Education, 6, 343-354.
28.
Gunter, P.L., & Denny, R.K. ( 1998). Trends and issues in research regarding academic instruction of students with emotional and behavioral disorders. Behavioral Disorders, 24, 44-50.
29.
Hawkins, J. ( 1988). Antecedent pausing as a direct instruction tactic for adolescents with severe behavior disorders. Behavioral Disorders , 13, 262-272.
30.
Hay, P.J., & MacDonald, D. ( 2008). (Mis)appropriations of criteria and standards-referenced assessment in a performance-based subject. Assessment in Education: Principles, Policy & Practice, 15, 153-168.
31.
Hockenbury, J.C., Kauffman, J.M., & Hallahan, D.P. (2000). What is right about special education? Exceptionality, 8(1), 3-12.
32.
Hodge, J., Riccomini, P.J., Buford, R., & Herbst, M.H. ( 2006). A review of instructional interventions in mathematics for students with behavioral disorders. Behavioral Disorders , 31, 297-311.
33.
Horner, R.H., Carr, E.G., Halle, J., McGee, G., Odom, S., & Wolery, M. ( 2005). The use of single subject design research to identify evidence-based practices in special education. Exceptional Children, 71, 165-179.
34.
Huberman, A.M., & Miles, M.B. ( 1994). Data management and analysis methods. In N. K. Denzin & Y. S. Lincoln (eds.), Handbook of qualitative research (pp. 428-444). Thousand Oaks, CA: Sage.
35.
IES What Works Clearinghouse. (2007, July 17). Beginning reading. Retrieved from http://ies.ed.gov/ncee/wwc/reports/beginning_reading/topic/
36.
Kauffman, J.M. ( 1999). Commentary: Today’s special education and its message for tomorrow. Journal of Special Education, 32, 244-255.
37.
Kazdin, A.E. ( 1982). Single-case research designs: Methods for clinical and applied settings. New York, NY: Oxford University Press.
38.
Keel, M.C., & Gast, D.L. ( 1992). Small-group instruction for students with learning disabilities: Observational and incidental learning. Exceptional Children , 58, 357-368.
39.
Lane, K.L., O’Shaughnessy, T.E., Lambros, K.M., Gresham, F.M., & Beebe-Frankenberger, M.E. (2001). The efficacy of phonological awareness training with first-grade students who have behavior problems and reading difficulties. Journal of Emotional and Behavioral Disorders, 9, 219-231.
40.
Law, M., Stewart, D., Pollock, N., Letts, L., Bosch, J., & Westmorland, M. (1998). Critical review form: Quantitative studies. Retrieved from http://www.fhs.mcmaster.ca/rehab/ebp/pdf/quanreview.pdf
41.
Law, M., Stewart, D., Pollock, N., Letts, L., Bosch, J., Westmorland, M., & Philpot, A. ( 2002). Occupational therapy evidence-based practice research group. Retrieved from http://srs-mcmaster.ca/Default.aspx?tabid=630
42.
Lincoln, Y.S., & Guba, E.G. ( 1985). Naturalistic inquiry. Beverly Hills, CA: Sage.
43.
Lonigan, C.J., Elbert, J.C., & Johnson, S.B. ( 1998). Empirically supported psychosocial interventions in children: An overview. Journal of Clinical Child Psychology, 27, 138-145.
44.
Maheady, L., Sacca, M.K., & Harper, G.F. ( 1988). Classwide peer tutoring with mildly handicapped high school students. Exceptional Children, 55, 52-59.
45.
Mattingly, J.C., & Bott, D.A. ( 1990). Teaching multiplication facts to students with learning problems. Exceptional Children, 56, 438-449.
46.
McEvoy, M.A., & Brady, M.P. ( 1988). Contingent access to play materials as an academic motivator for autistic and behavior disordered children. Education and Treatment of Children, 4, 1-15.
47.
McKenzie, M.L., & Budd, K.S. ( 1981). A peer tutoring package to increase mathematics performance: An examination of generalized changes in classroom behavior. Education and Treatment of Children, 4, 1-15.
48.
McLaughlin, T.F. ( 1991). Use of a personalized system of instruction with and without a same-day retake contingency on spelling performance of behaviorally disordered children. Behavioral Disorders, 16, 127-132.
49.
McLaughlin, T.F. ( 1992). Effects of written feedback in reading on behaviorally disordered students. Journal of Educational Research, 85, 312-316.
50.
Mooney, P., Epstein, M.H., Reid, R., & Nelson, J.R. ( 2003). Status and trends in academic intervention research for students with emotional disturbance. Remedial and Special Education, 24, 273-287.
51.
Odom, S. ( 2007). QI’s for research and evidence-based practice in special education. Retrieved from http://www.cecdr.org/news.cfm?id=FD48BC67-1422-201A-559D77EEA2AEC4E0
52.
Odom, S.L., Brantlinger, E., Gersten, R., Horner, R., Thompson, B., & Harris, K. ( 2005). Research in special education: Scientific methods and evidence-based practices. Exceptional Children, 71, 137-148.
53.
Odom, S.L., Brown, W.H., Frey, T., Karasu, N., Smith-Canter, L.L., & Strain, P.S. ( 2003). Evidence-based practices for young children with autism: Contributions for single-subject design research. Focus on Autism and Other Developmental Disabilities, 18, 166-175.
54.
Odom, S.L., Brown, W.H., Frey, T., Karasu, N., Smith-Canter, L.L., & Strain, P.S. (n.d.). Part B: Specialized coding sheet for single subject studies. Unpublished form.
55.
Odom, S.L., & Strain, P.S. ( 2002). Evidence-based practice in early intervention/early childhood special education: Single-subject research design. Journal of Early Intervention, 25, 151-160.
56.
Ollendick, T.H., Matson, J.L., Esveldt-Dawson, K., & Shapiro, E. ( 1980). Increasing spelling achievement: An analysis of treatment procedures utilizing an alternating treatments design. Journal of Applied Behavioral Analysis, 13, 645-654.
57.
Peters, M.T., & Heron, T.E. ( 1993). When the best is not good enough: An examination of best practice. Journal of Special Education, 26, 371-385.
58.
Pierce, C.D., Reid, R., & Epstein, M.H. ( 2004). Teacher-mediated interventions for children with EBD and their academic outcomes. Remedial and Special Education , 25, 175-188.
59.
Reichow, B., Volkmar, F.R., & Cicchetti, D.V. (2008). Development of the evaluative method for evaluating and determining evidence-based practices in autism. Journal of Autism and Developmental Disorders, 38, 1311-1319.
60.
Reid, R., Gonzalez, J.E., Nordness, P.D., Trout, A., & Epstein, M.H. ( 2004). A meta-analysis of the academic status of students with emotional/behavioral disturbance. Journal of Special Education , 38, 130-143.
61.
Roach, A.T., Elliott, S.N., & Webb, N.L. ( 2005). Alignment of an alternate assessment with state academic standards: Evidence for the content validity of the Wisconsin alternate assessment . Journal of Special Education, 38, 218-231.
62.
Rose, T.L. ( 1984). Effects of previewing on the oral reading of mainstreamed behaviorally disordered students. Behavioral Disorders, 10, 33-39.
63.
Ruhl, K.L., & Berlinghoff, D.H. (1992). Research on improving behaviorally disordered students’ academic performance: A review of the literature . Behavioral Disorders, 17, 178-190.
64.
Ryan, J.B., Reid, R., & Epstein, M.H. ( 2004). Peer-mediated intervention studies on academic achievement for students with EBD: A review. Remedial and Special Education , 25, 330-341.
65.
Schloss, P.J., Harriman, N.E., & Pfeifer, K. ( 1985). Application of sequential prompt reduction technique to the independent composition performance of behaviorally disordered youth . Behavioral Disorders, 12, 17-23.
66.
Shapiro, E.S., & McCurdy, B.L. ( 1989). Effects of taped-words treatment on reading proficiency . Exceptional Children, 55, 321-325.
67.
Singer, G.H.S. ( 2000). Special education research agenda in the United States: Sources of influence. Exceptionality, 8(1), 235-247.
68.
Skinner, C.H., Bamberg, H.W., Smith, E.S., & Powell, S.S. ( 1993). Cognitive cover, copy, and compare: Subvocal responding to increase rates of accurate division responding. Remedial and Special Education, 14, 49-56.
69.
Skinner, C.H., Beatty, K.L., Turco, T.L., & Rasavage, C. ( 1989). Cover, copy and compare: A method for increasing multiplication performance. School Psychology Review, 18, 412-420.
70.
Skinner, C.H., Belfiore, P.J., Mace, H.W., Williams-Wilson, S., & Johns, G.A. ( 1997). Alternating response topography to increase response efficiency and learning rates. School Psychology Quarterly , 12, 54-64.
71.
Skinner, C.H., Belfiore, P.J., & Pierce, N. ( 1992). Cover, copy, and compare: Increasing geography accuracy in students with behavior disorders. School Psychology Review , 21, 73-81.
72.
Smith, B.J., Strain, P.S., Snyder, P., Sandall, S.R., McLean, M.E., Broudy Ramsey, A., & Sumi, W.C. ( 2002). DEC recommended practices: A review of 9 years of EI/ECSE research literature. Journal of Early Intervention, 25, 108-119.
73.
Spooner, F., & Browder, D.M. ( 2003). Scientifically based research in education and students with low incidence disabilities. Research and Practice for Persons with Severe Disabilities, 28, 117-125.
74.
Stemler, S. ( 2001). An overview of content analysis. Practical Assessment , Research and Evaluation, 7, 213-225.
75.
Stowitschek, C.E., Hecimovic, A., Stowitschek, J.J., & Shores, R.E. ( 1982). Behaviorally disordered adolescents as peer tutors: Immediate and generative effects on instructional performance and spelling achievement . Behavioral Disorders, 7, 136-148.
76.
Stowitschek, C.E., Lewis, B.L., Shores, R.E., & Ezzell, D.L. ( 1980). Procedures for analyzing student performance data to generate hypotheses for the purpose of educational decision making. Behavioral Disorders, 5, 136-150.
77.
Sutherland, K.S., Alder, N., & Gunter, P.L. ( 2003). The effects of varying rates of opportunities to respond to academic requests on the classroom behavior of students with EBD. Journal of Emotional and Behavioral Disorders, 11, 239-248.
78.
Sutherland, K.S., & Singh, N.N. ( 2004). Learned helplessness and students with emotional or behavioral disorders: Deprivation in the classroom. Behavioral Disorders , 29, 169-181.
79.
Sutherland, K.S., & Wehby, J.H. ( 2001). Exploring the relationship between increased opportunities to respond to academic requests and the academic and behavioral outcomes of student with EBD. Remedial and Special Education, 22, 115-121.
80.
Swanson, H.L., Hoskyn, M., & Lee, C. ( 1999). Interventions for students with learning disabilities: A meta-analysis of treatment outcomes. New York, NY: Guilford.
81.
Swanson, H.L., & Sachse-Lee, C. (2000). A meta-analysis of single-subject-design intervention research for students with LD. Journal of Learning Disabilities, 33, 114-136.
82.
Tankersley, M., & Cook, B.G. ( 2007). Using QI’s to identify quality research. Retrieved from http://www.cecdr.org/news.cfm?id=FD48BC67-1422-201A-559D77EEA2AEC4E0
83.
Test, D.W., Fowler, C.H., Brewer, D.M., & Wood, W.M. ( 2005). A content and methodological review of self-advocacy intervention studies. Exceptional Children, 72, 101-125.
84.
Trochim, W.M.K. ( 2001). The research methods knowledge base (2nd ed.). Cincinnati, OH: Atomic Dog .
85.
Trout, A.L., Nordness, P.D., Pierce, C.D., & Epstein, M.H. ( 2003). Research on the academic statues of children with emotional and behavioral disorders: A review of the literature from 1961 to 2000. Journal of Emotional and Behavioral Disorders, 11, 198-210.
86.
U.S. Department of Education. (2001). Twenty-second annual report to Congress on the implementation of P.L. 94-142 . Washington, DC: Author.
87.
U.S. Department of Education. (2005, December). IES style guide. Retrieved from http://nces.ed.gov/statprog/styleguide/pdf/styleguide.pdf
88.
Wolery, M., Cybriwsky, C.A., Gast, D.L., & Boyle-Gast, K. (1991). Use of constant time delay and attentional responses with adolescents. Exceptional Children , 57, 464-474.
