Two different theoretical approaches were compared to determine the optimal cutoff scores for the Ages and Stages Questionnaires: Social-Emotional (ASQ: SE), a social-emotional screening test. Cutoff scores based on statistical decision theory modeling, Receiver Operator Characteristics (ROC), were compared with cutoff scores obtained using Item Response Theory (IRT) measurement theory modeling. The purposes of this comparison were (a) to offer alternative perspectives on the computation and function of cutoff scores, and (b) to obtain appropriate cutoff scores for the ASQ: SE screening test. Results are presented for both analyses, ROC and IRT Collectively, results from the two approaches offer comparable solutions to persistent problems arising when screening decision rules are needed. Results underscore the psychometric integrity of the ASQ: SE cutoff scores.
References
1.
Achenbach, T. (1991). Child behavior checklistl2-3 years (CBCL/2-3) . Burlington, VT: University of Vermont, Department of Psychiatry.
2.
Achenbach, T. (1992). Child behavior checklistl4-18 (CBLC/4-18). Burlington, VT: University of Vermont, Department of Psychiatry.
3.
American Psychological Association. (1999). Standards for educational and psychological testing. American Educational Research Association and National Council on Measurement in Education .
4.
Andersson, L. (2004). Appropriate and inappropriate interpretation and use of test scores in early intervention. Journal of Early Intervention, 27, 55-68.
5.
Andrich, D. (1978). Application of a psychometric model to ordered categories which are scored with successive integers. Applied Psychological Measurement, 2, 581-594.
6.
Aylward, G. (1997). Conceptual issues in developmental screening and assessment. Journal of Developmental and Behavioral Pediatrics , 18, 340-349.
7.
Bagnato, S., & Neisworth, J. (1999). Collaboration and teamwork in assessment for early intervention. Comprehensive Psychiatric Assessment of Young Children, 8, 347-363.
8.
Bricker, D., & Squires, J. (1999). Ages & stages questionnaires: A parent-completed, child-monitoring system (2nd ed.). Baltimore: Paul Brookes.
9.
Dobrez, D., Sasso, A., Holl, J., Shalowitz, M., Leon, S., & Budetti, P. (2001). Estimating the cost of developmental and behavioral screening of preschool children in general pediatric practice. Pediatrics, 108, 913-922.
10.
Embretson, S.E., & Reise, S.P. (2000). Item response theory for pyschologists. Mahwah, NJ: Lawrence Erlbaum Associates .
11.
Feldt, L.S., & Brennan, R.L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). New York: American Council on Education and Macmillan.
12.
Feldt, L.S., & Qualls, A.L. (1996). Estimation of measurement error variance at specifica score levels. Journal of Educational Measurement, 33, 141-156.
13.
Fleiss, J. (1986). The design and analysis of clinical experiments. (Wiley series in probability and mathematical statistics: Applied probability and statistics). Australia: John Wiley & Sons.
14.
Hambleton, R.K., & Swaminathan, H. (1985). Item response theory principles and applications. Boston: Kluwer Nijhoff Publishing.
15.
Kolen, M., Zeng, L., & Hanson, B. (1996). Conditional standard errors of measurement for scale scores using IRT. Journal of Educational Measurement , 33, 129-140.
16.
Linacre, M. (2006). WINSTEPS Version 3.61 [Computer Software]. Chicago: MESA Press.
17.
Lord, F. (1980). Applications of item response theory to practical testing problems. Hillsdale, NJ: Lawrence Erlbaum Associates.
18.
MacDonald, P., & Paunenon, S.V. (2002). A monte carlo comparison of item and person statistics based on item response theory versus classical test theory. Educational and Psychological Measurement, 62, 921-943.
19.
Reynolds, A., Ou, S., & Topitzes, J. (2004). Paths of effects of early childhood intervention on educational attainment and delinquencey: A confirmatory analysis of the Chicago child-parent centers. Child Development, 75, 1299-1328.
20.
Shonkoff, J., & Phillips, D. (Eds.) (2000). From neurons to neighborhoods: The science of early childhood development. Washington, DC: Academic Press.
21.
Smith, R. (2000). Fit analysis in latent trait measurement models . Journal of applied measurement, 1, 199-218.
22.
Sparrow, S., Balla, D., & Cicchetti, D. (1998). Vineland social-emotional early childhood scales . Circle Pines, MN: American Guidance Service.
23.
Squires, J., Bricker, D., Heo, K., & Twombly, E. (2001). Identification of social-emotional problems in young children using a parent-completed screening measure. Early Childhood Research Quarterly, 16, 405-419.
24.
Squires, J., Bricker, D., & Potter, L. (1999). The ASQ user's guide for the Ages & Stages Questionnaires: A Parent-Completed, Child-Monitoring System (2nd ed.). Baltimore: Brookes.
25.
Squires, J., Bricker, D., & Twombly, E. (2002). Ages and stages questionnaires: Social-emotional . Baltimore: Brookes.
26.
Squires, J., Bricker, D., & Twombly, E. (2004). Parent-completed screening for social-emotional problems in young children: The effects of risk/disability status and gender on performance. Infant Mental Health Journal, 25, 62-73.
27.
Swets, J. (1986). Form of empirical ROCs in discrimination and diagnostic tasks: Implications for theory and measurement of performance. Psychological Bulletin, 99, 181-198.
28.
Swets, J., & Picket, R. (1982). Evaluation of diagnostic systems: Methods from signal detection theory. New York: Academic Press.
29.
Swets, J., Dawes, R., & Monahan, J. (2000). Psychological science can improve diagnostic decisions . Psychological Science in the Public Interest, 1.
30.
Tuzlukov, & Vyacheslav, P. (2001). Signal detection theory. New York: Springer Verlag.
31.
U.S. Bureau of Census. (2000). Retrieved on July 31, 2000 at www.census.gov/
32.
Van der Linden, W. J., & Hambleton, R. K. (Eds.) (1997). Handbook of modern item response theory. New York: Springer.
33.
Walker, H., Stiller, B., Severson, H., Feil, E., & Golly, A. (1998). First step to success: Intervening at the point of school entry to prevent antisocial behavior patterns. Psychology in the Schools, 35, 259-269.
34.
Zeanah, C. (Ed.) (2000). Handbook of infant mental health (2nd ed.). New York: Guilford.
