This study investigated the equivalence of scores from computerized and paper-and-pencil formats of a series of K-3 reading screening tests. Concerns about score equivalence on the computerized formats were warranted because of the use of reading passages, computer unfamiliarity of primary school students, and teacher versus computer administration of the test. The results indicated that the computerized format produced lower scores than the paper-and-pencil format across all four grades. These difficulty differences could be related to the differences in family income but further research is needed into the causal factors. However, confirmatory factor analysis supported the construct and measurement equivalence of the scores from the two formats.
References
1.
American Psychological Association. (1986). Guidelines for computer-based tests and interpretations. Washington, D.C.: Author.
2.
BarnettV., & LewisT. (1984). Outliers in statistical data. New York: Wiley.
3.
BugbeeA. C. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28(3), 282–290.
4.
ChoiS. W., & TinklerT. (2002, April). Evaluating comparability of paper-and-pencil and computer-based assessment in a K-12 setting. Paper presented at the annual meeting of AERA, New Orleans.
5.
ClansingC., & SchmittD. (1990). Paper versus CRT: Are reading rate and comprehension affected? Paper available from ERIC.
6.
CohenJ. (1988). Statistical power analysis for the behavioral sciences. Hillsdale, NJ: Lawrence Erlbaum Associates.
FeldmannS. C., & FishM. C. (1987, April). Reading comprehension of high school students on print vs. microcomputer-generated text. Paper presented at the annual meeting of the American Educational Research Association, Washington, D.C.
10.
FleishmanJ., & BensonJ. (1987). Using LISREL to evaluate measurement models and scale reliability. Educational and Psychological Measurement, 47, 925–939.
11.
FuchsD., & FuchsL. S. (1989). Effects of examiner familiarity on Black, Caucasian, and Hispanic children: A meta-analysis. Exceptional Children, 55, 303–308.
12.
GlassG. V., & HopkinsK. D. (1996). Statistical methods in education and psychology. Boston: Allyn and Bacon.
13.
GredlerG. R. (1992). School readiness: Assessment and educational issues. Brandon, VT: Clinical Psychology Publishing Company, Inc.
14.
HuL., & BentlerP.M. (1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1–55.
15.
KielyG. L.ZaraA. R., & WeissD. J. (1986). Equivalence of computer and paper-and-pencil Armed Services Vocational Aptitude Battery Tests. Research Report No. AFHRL-TP-86–13. Air Force Human Resources Laboratory, Brooks Air Force Base, TX.
16.
LeeJ.MorenoK. E., & SympsonJ. B. (1986). The effects of mode of test administration on test performance. Educational and Psychological Measurement, 46, 467–473.
17.
MarshH. W. (1994). Confirmatory factor analysis models of factorial invariance: A multi-faceted approach. Structural Equation Modeling, 1, 5–34.
18.
MazzeoJ., & HarveyA. (1988). The equivalence of scores from automated and conventional educational and psychological tests. College Board Report 88–8. New York, College Board.
19.
NeumanG., & BaydounR. (1998). Computerization of paper-and-pencil tests: When are they equivalent?Applied Psychological Measurement, 22(1), 71–83.
20.
NunnallyJ. C. (1978). Psychometric Theory. New York: McGraw-Hill.
21.
OlsenJ. B.MaynesD. D.SlawsonD., & HoK. (1989). Comparison of paper-administered, computer-administered and computerized adaptive achievement tests. Journal of Educational Computing Research, 5, 311–326.
22.
Riverside Publishing Co. (2002). Basic early assessment of reading. Itasca, IL: Riverside Publishing Co.
23.
Riverside Publishing Co. (2003). Technical manual—Basic early assessment of reading.Itasca, IL: Riverside Publishing Co.
24.
RozeboomW. W. (1966). Foundations of the theory of prediction. Homewood, IL: Dorsey.
25.
Sutton (1991). Equity and computers in schools: A decade of research. Review of Educational Research, 61(4), 475–503.
26.
U.S. Department of Education, National Center for Education Statistics (2002). Digest of education statistics 2001, NCES 2000–130, Washington, D.C.: Author.
27.
WiseS. L.BarnesL. B.HarveyA., & PlakeB. S. (1989). The effects of computer anxiety and computer experience on the computer-based achievement test performance of college students. Applied Measurement in Education, 2, 235–241.
