This article develops two procedures for estimating individual-level conditional standard errors of measurement (SEMs) for scale scores, assuming tests consist of dichotomously scored items. Using binomial model assumptions, one procedure provides a scale-score analogue of Lord’s raw-score SEM. Assuming a compound binomial model, another procedure provides a scale-score analogue of Feldt’s raw-score SEM. These two procedures are compared to a polynomial procedure and a procedure developed by Feldt and Qualls using data from the Iowa Tests of Basic Skills.
References
1.
American Educational Research Association , American Psychological Association, & National Council on Measurement in Education. (1985). Standards for educational and psychological testing. Washington, DC: American Psychological Association.
2.
Brennan, R. L. (1992). Elements of generalizability theory (Rev. ed.). Iowa City, IA: American College Testing.
3.
Brennan, R. L. (1996). Conditional standard errors of measurement in generalizability theory (Iowa Testing Programs Occasional Paper No. 40). Iowa City: University of Iowa.
4.
Brennan, R. L. (1998). Raw-score conditional standard errors of measurement in generalizability theory. Applied Psychological Measurement, 22, 307-331.
5.
Brennan, R. L. , & Lee, W. (1997). Conditional standard errors of measurement for scale scores using binomial and compound binomial assumptions (Iowa Testing Programs Occasional Paper No. 41). Iowa City: University of Iowa.
6.
Colton, D. A. , Gao, X., & Kolen, M. J. (1996, April). Assessing the reliability of performance level scores using bootstrapping. Paper presented at the annual meeting of the American Educational Research Association, New York.
7.
Feldt, L. S. (1984). Some relationships between the binomial error model and classical test theory. Educational and Psychological Measurement, 44, 883-891.
8.
Feldt, L. S. , & Brennan, R. L. (1989). Reliability. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 105-146). New York: American Council on Education.
9.
Feldt, L. S. , & Qualls, A. L. (1996). Estimation of measurement error variance at specific score levels. Journal of Educational Measurement, 33, 141-156.
10.
Feldt, L. S. , & Qualls, A. L. (1998). Approximating scale score standard error of measurement from the raw score standard error. Applied Measurement in Education, 11, 159-177.
11.
Hoover, H. D. , Hieronymus, A. N., Frisbie, D. A., & Dunbar, S. B. (1993). Iowa tests of basic skills (Levels 9-14). Chicago: Riverside.
12.
Jarjoura, D. , & Brennan, R. L. (1982). A variance components model for measurement procedures associated with a table of specifications. Applied Psychological Measurement, 6, 161-171.
13.
Kendall, M. G. , & Stuart, A. (1977). The advanced theory of statistics (4th ed., Vol. 1). New York: Macmillan.
14.
Kolen, M. J. , & Brennan, R. L. (1995). Test equating: Methods and practices. New York: Springer-Verlag.
15.
Kolen, M. J. , Hanson, B. A., & Brennan, R. L. (1992). Conditional standard errors of measurement for scale scores. Journal of Educational Measurement, 29, 285-307.
16.
Kolen, M. J. , Zeng, L., & Hanson, B. A. (1996). Conditional standard errors of measurement for scale scores using IRT. Journal of Educational Measurement, 33, 129-140.
17.
Lord, F. M. (1955). Estimating test reliability. Educational and Psychological Measurement, 15, 325-336.
18.
Lord, F. M. (1957). Do tests of the same length have the same standard errors of measurement?Educational and Psychological Measurement, 17, 510-521.
19.
Lord, F. M. (1965). A strong true score theory, with applications. Psychometrika, 30, 239-270.
20.
Mollenkopf, W. G. (1949). Variation of the standard error of measurement. Psychometrika, 14, 189-229.
21.
Thorndike, R. L. (1951). Reliability. In E. F. Lindquist (Ed.), Educational measurement (pp. 560-620). Washington, DC: American Council on Education.
22.
Wang, T. , Kolen, M. J., & Harris, D. J. (1996, April). Conditional standard errors, reliability and decision consistency of performance levels using polytomous IRT. Paper presented at the annual meeting of the American Educational Research Association, New York.
