Textbook writers often recommend four or five options per multiple-choice item, and most, if not all, testing programs in the United States also employ four or five options. Recent reviews of research on the desirable number of options for a multiple-choice test item reveal that three options may be suitable for most ability and achievement tests. A study of the frequency of acceptably performing distractors is reported. Results from three different testing programs support the conclusion that test items seldom contain more than three useful options. Consequently, testing program personnel and classroom teachers may be better served by using 2-or 3-option items instead of the typically recommended 4- or 5-option items.
References
1.
Budescu, D. V. and Nevo, B. (1985). Optimal number of options: An investigation of the assumption of proportionality. Journal of Educational Measurement, 22, 183-196.
2.
Cronbach, L. J. (1970). Review of "On the theory of achievement test items" by J. R. Bormuth. Psychometrika, 35, 509-511.
3.
Downing, S. M. (1992). True-false and alternative-choice formats: A review of research. Educational Measurement: Issues and Practices, 11, 27-30.
4.
Ebel, R. L. (1969). Expected reliability as a function of choices per item. Educational and Psychological Measurement, 29, 565-570.
5.
Ebel, R. L. and Frisbie, D. A. (1986). Essentials of educational measurement (4th ed.) Englewood, Cliffs, NJ: Prentice-Hall.
6.
Grier, J. B. (1976). The optimal number of alternatives at a choice point with travel time considered. Journal of Mathematical Psychology, 14, 91-97.
7.
Haladyna, T. M. and Downing, S. M. (1989a) A taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2, 37-50.
8.
Haladyna, T. M. and Downing, S. M. (1989b). The validity of a taxonomy of multiple-choice item-writing rules. Applied Measurement in Education, 2, 51-78.
9.
Haladyna, T. M. and Sympson, J. B. (1988). Paper presented at the symposium entitled "Empirically based polychotomous scoring of multiple-choice test items: A review" in C. E. Davis (Chair), New developments in polychotomous scoring and modeling. Symposium conducted at the annual meeting of the American Educational Research Association, New Orleans, LA.
10.
Levine, M. V. and Drasgow, F. (1983). The relation between incorrect option choice and estimated ability. Educational and Psychological Measurement, 43, 675-685.
11.
Lord, F. M. (1944). Reliability of multiple-choice tests as a function of number of choices per item. Journal of Educational Psychology, 35, 175-180.
12.
Lord, F. M. (1977). Optimal number of choices per item-A comparison of four approaches. Journal of Educational Measurement, 14, 33-38.
13.
Nitko, A. J. (1989). Designing tests that are integrated with instruction. In R. L. Linn (Ed.) Educational Measurement (3rd ed., pp. 447-474). Washington, DC: MacMillan.
14.
Owen, S. V. and Froman, R. D. (1987). What's wrong with three-option multiple-choice items?Educational and Psychological Measurement, 47, 513-522.
15.
Thissen, D. M. , Steinberg, L., and Fitzpatrick, A. R. (1989). Multiple-choice models: The distractors are also part of the item. Journal of Educational Measurement, 26, 161-176.
16.
Trevisan, M. S. , Sax, G., and Michael, W. B. (1991). The effects of the number of options per item and student ability on test validity and reliability. Educational and Psychological Measurement, 51, 829-837.
17.
Tversky, A. (1964). On the optimal number of alternatives at a choice point. Journal of Mathematical Psychology, 1, 386-391.
