The relative merits and potential of item transformations and the cloze procedure are evaluated. The two strategies for assessing reading comprehension were reviewed in terms of three criteria: (1) the extent to which literal comprehension skills are validly measured; (2) the extent to which the issue of passage dependency is addressed; (3) utility for classroom teachers, reading specialists, and evaluators of reading programs. The analysis revealed that the cloze procedure offers several distinct advantages over six sentence-derived transformations. They include the assessment of both semantic and syntactic relationships, dependency on contextual clues, and ease of development, administration, and scoring. Both item generation methods were applicable to only content domains that could be decomposed into passages and sentences. Unfortunately, this limitation encompasses many of the functional skills and minimum competencies that are in the process of being measured in 38 states.
References
1.
AllingtonR. L.ChodosL.DomarackiJ.TruexS.Passage-dependency: Four diagnostic oral reading tests. The Reading Teacher, 1977, 30, 369–375.
2.
AndersonR. C.How to construct achievement tests to assess comprehension. Review of Educational Research, 1972, 42, 145–170.
3.
BerkR. A.Estimation of test length for domain-referenced reading comprehension tests. Journal of Experimental Education, in press.
4.
BickleyA. C.EllingtonB. J.BickleyR. T.A cloze procedure: A conspectus. Journal of Reading Behavior, 1970, 2, 232–243.
5.
BormuthJ. R.Optimum sample size and cloze test length in readability measurement. Journal of Educational Measurement, 1965, 2, 111–116.
6.
BormuthJ. R.On the theory of achievement test items. Chicago: University of Chicago Press, 1970.
7.
ChomskyN.Aspects of the theory of syntax. Cambridge, MA: The MIT Press, 1965.
8.
ClymerT.What is reading? Some current concepts. In RobinsonH. B. (Ed.), Innovation and change in reading instruction: The sixty-seventh yearbook of the National Society for the Study of Education: Part II. Chicago: University of Chicago Press, 1968. Pp. 19–23.
9.
CranneyA.A comparison of a free-response and multiple-choice cloze reading test. Unpublished doctoral dissertation, University of Minnesota, 1968.
10.
CronbachL. J.Review of Bormuth's On the theory of achievement test items. Psychometrika, 1970, 35, 509–511.
11.
DeckD.Validity of word deletion items as a measure of reading comprehension. Paper presented at the annual meeting of the American Educational Research Association, New York, April 1977.
12.
DiederichP. B.Review of Bormuth's On the theory of achievement test items. Educational and Psychological Measurement, 1970, 30, 1003–1005.
13.
FarrR.SmithC. B.The effects of test item validity on total test reliability and validity. In SchickG. B.MayM. M. (Eds.), Reading: Process and pedagogy. Milwaukee, WI: National Reading Conference, 1970. Pp. 122–134.
FinnP. J.Generating domain-referenced, multiple-choice test items from prose passages. Paper presented at the annual meeting of the American Educational Research Association, Toronto, March 1978.
16.
GuthrieJ. T.SeigerM.BurnhamH.KaplanR.The maze technique to assess, monitor reading comprehension. The Reading Teacher, 1974, 28, 161–169.
17.
HerberH.Teaching reading in content areas. Englewood Cliffs, NJ: Prentice-Hall, 1970.
18.
JohnsJ. L.Do comprehension items really test reading? Sometimes!Journal of Reading, 1978, 21, 615–619.
19.
LordF. M.Sampling fluctuations resulting from the sampling of test items. Psychometrika, 1955, 20, 1–22.
20.
MehrensW. A.Scientific test construction: Pure and sterile (Review of Bormuth's On the theory of achievement test items). Contemporary Psychology. 1970, 15, 666–667.
21.
MenzelP.The linguistic bases of the theory of writing items for instruction stated in natural language. In BormuthJ. R., On the theory of achievement test items. Chicago: University of Chicago Press, 1970. Pp. 87–155.
22.
OhnmachtF. W.WeaverW. W.KohlerE. T.Cloze and closure: A factoral study. Journal of Psychology, 1970, 74, 205–217.
23.
O'reillyR. P.StreeterR. E.Reports on the development and validation of a system for measuring literal comprehension in a multiple-choice cloze format: Preliminary factor analytic results. Journal of Reading Behavior, 1977, 9, 45–49.
24.
PerfettiC. A.Psychosemantics: Some cognitive aspects of structured meaning. Psychological Bulletin, 1972, 78, 241–259.
25.
PrestonR. C.Ability of students to identify correct responses before reading. Journal of Educational Research, 1964, 58, 181–183.
26.
PyrczakF.Objective evaluation of the quality of multiple-choice test items designed to measure comprehension of reading passages. Reading Research Quarterly, 1972, 8, 62–71.
27.
PyrczakF.Passage-dependence of items designed to measure the ability to identify the main ideas of paragraphs: Implications for validity. Educational and Psychological Measurement, 1974, 34, 343–348.
28.
PyrczakF.Passage-dependence of reading comprehension questions: Examples. Journal of Reading, 1975, 18, 308–311.
29.
PyrczakF.A responsive note on measures of the passage-dependence of reading comprehension test items. Reading Research Quarterly, 1975–76, 11, 112–117.
30.
PyrczakF.Context-independence of items designed to measure the ability to derive meaning of words from their context. Educational and Psychological Measurement, 1976, 36, 919–924.
31.
PyrczakF.AxelrodJ.Determining passage dependence of reading comprehension exercises: A call for replications. Journal of Reading, 1976, 18, 279–283.
32.
RankinE. F.An evaluation of the cloze procedure as a technique for measuring reading comprehension. Unpublished doctoral dissertation, University of Michigan, 1957.
33.
RankinE. F.A definition of reading comprehension. First yearbook of the North Central Reading Association. Minneapolis: University of Minnesota, 1959. (a).
34.
RankinE. F.Uses of the cloze procedure in the reading clinic. In FigurelJ. A. (Ed.), International Reading Association Conference Proceedings. New York: Scholastic Magazines, 1959. (b).
35.
RankinE. F.Closure and the cloze procedure. Third yearbook of the North Central Reading Association. Minneapolis: University of Minnesota, 1964.
36.
RankinE. F.Cloze procedure—a survey of research. In ThurstonE. L.HafnerL. E. (Eds.), Fourteenth yearbook of the National Reading Conference. Milwaukee: National Reading Conference, 1965.
37.
RankinE. F.The cloze procedure—Its validity and utility. In FarrR. (Ed.), Measurement and evaluation of reading. New York: Harcourt, Brace & World, 1970. Pp. 237–253.
38.
RansonP.Determining reading levels of elementary school children by cloze testing. In FigurelJ. A. (Ed.), Forging ahead in reading. Newark, DE: International Reading Association, 1968.
39.
RichekM. A.Reading comprehension of anaphoric forms in varying linguistic courses. Reading Research Quarterly, 1976–77, 12, 145–165.
40.
RobinsonR. D.An introduction to the cloze procedure: An annotated bibliography. Newark, DE: International Reading Association, 1972.
41.
RoidG. H.HaladynaT. M.Comparison of objective-based and modified-Bormuth item writing techniques. Educational and Psychological Measurement, 1978, 38, 19–28.
42.
RothkopfE. Z.Textual constraint as a function of repeated inspection. Journal of Educational Psychology, 1968, 59, 20–25.
43.
TaylorW. L.Cloze procedure: A new tool for measuring readability. Journalism Quarterly, 1953, 30, 415–433.
44.
TaylorW. L.Application of “cloze” and entropy measures to the study of contextual constraints in samples of continuous prose. Unpublished doctoral dissertation, University of Illinois, 1954.
45.
TaylorW. L.Recent developments in the use of the cloze procedure. Journalism Quarterly, 1956, 33, 42–48.
46.
TaylorW. L.“Cloze” readability scores as indices of individual differences in comprehension and aptitude. Journal of Applied Psychology, 1957, 41, 19–26.
47.
TuinmanJ. J.Inspection of passages as a function of passage-dependency of the test items. Journal of Reading Behavior, 1972–73, 5, 186–192.
48.
TuinmanJ. J.Determining the passage dependence of comprehension questions in five major tests. Reading Research Quarterly, 1973–74, 9, 206–223.
49.
WeaverW. W.BickleyA. C.FordF.A cross-validation study of the relationship of reading test items to their relevant paragraphs. Perceptual and Motor Skills, 1969, 29, 11–14.
