Examining the Agreement of Direct Behavior Ratings and Systematic Direct Observation Data for On-Task and Disruptive Behavior

Abstract

The purpose of this study was to replicate previous findings indicating a moderate association between teacher perceptions of behavior as measured by direct behavior ratings (DBRs) and systematic direct observation (SDO) conducted by an external observer. In this study, data regarding student on-task and disruptive behavior were collected via SDO from trained external observers and via DBRs from classroom teachers. Data were collected across 15 teachers and three observation sessions, and the agreement between the two methods was compared as a way to examine concurrent validity. Results supported previous work suggesting that DBRs are significantly correlated with SDO data, thereby suggesting that the DBR might be used as a compatible tool with SDO. Implications for practice, limitations of the study, and directions for future research are discussed.

Keywords

social behavior assessment direct behavior ratings

References

Alberto, P.A. , & Troutman, A.C. (1999). Applied behavior analysis for teachers. Upper Saddle River, NJ: Merrill.

Chafouleas, S.M. , McDougal, J.L. , Riley-Tillman, T.C. , Panahon, C.J. , & Hilt, A.M. (2005). What do daily behavior report cards (DBRCs) measure? An initial comparison of DBRCs with direct observation for off-task behavior . Psychology in the Schools, 42, 669-676.

Chafouleas, S.M. , Riley-Tillman, T.C. , & McDougal, J. (2002). Good, bad, or in-between: How does the daily behavior report card rate? Psychology in the Schools, 39, 157-169.

Chafouleas, S.M. , Riley-Tillman, T.C. , & Sassu, K.A. (in press). An investigation of the reported acceptability and usage of daily behavior report cards (DBRCs) by teachers. Journal of Positive Behavior Interventions.

Cone, J.D. (1978). The behavioral assessment grid (BAG): A conceptual framework and a taxonomy. Behavior Therapy, 9, 882-888.

Fuchs, L.S. , & Fuchs, D. (2003). What is scientifically-based research on progress monitoring? Retrieved November 15, 2003, from http://www.edformation.com/Articles.htm

Guttman, L. (1950). The basis for scalogram analysis. In S. A. Stouffer , L. Guttman , E. A. Suchman , P. F. Lazarsfeld , S. A. Star , & J. A Claussen (Eds.), Measurement and prediction (pp. 60-90). Princeton, NJ: Princeton University Press.

Hintze, J.M. (2005). Psychometrics of direct observation. School Psychology Review, 34, 507-519.

Kehle, T.J. , Clark, E. , & Jenson, W.R. (1986). Effectiveness of self-observation with disordered elementary school children. School Psychology Review, 15, 289-295.

10.

Irvin, L.K. , Tobin, T.J. , Sprague, J.R. , Sugai, G. , & Vincent, C.G. (2004). Validity of office discipline referral measures as indices of school-wide behavioral status and effects of school-wide behavioral interventions. Journal of Positive Behavior Interventions , 6, 131-147.

11.

McCain, A.P. , & Kelly, M.L. (1993). Managing the classroom behavior of an ADHD preschooler: The efficacy of a school- home note intervention. Child and Family Behavior Therapy, 15, 33-44.

12.

Nolan, E.E. , & Gadow, K.D. (1994). Relation between ratings and observations of stimulant drug response in hyperactive children. Journal of Clinical Child Psychology, 23, 78-90.

13.

Pelham, W.E. (1993). Pharmacotherapy for children with attention deficit hyperactivity disorder. School Psychology Review, 22, 199-227.

14.

Riley-Tillman, T.C. , Kalberer, S.M. , & Chafouleas, S.M. (2005). Selecting the right tool for the job: A review of behavior monitoring tools used to assess student response-to-intervention. California School Psychologist, 10, 81-92.

15.

Saudargas, R.A. , & Lentz, F.E. (1986). Estimating percent of time and rate via direct observation: A suggested observation procedure and format. School Psychology Review, 15(1), 36-48.

16.

Steege, M.W. , Davin, T. , & Hathaway, M. (2001). Reliability and accuracy of a performance-based behavioral recording procedure. School Psychology Review, 30, 252-261.

17.

Turco, T.L. , & Elliott, S.N. (1986). Students acceptability ratings of interventions for classroom misbehaviors: A study of well-behaving and misbehaving youth . Journal of Psychoeducational Assessment, 4, 281-289.

18.

Wright, J.A. , & Dusek, J.B. (1998). Compiling base rates for disruptive behaviors from student discipline referral data. School Psychology Review , 27, 147-183.