Eelworms,Bullet Holes,and Geraldine Ferraro: Some Problems With Statistical Adjustment and Some Solutions

There is no safety in numbers. When data are gathered from a sample in which the selection criteria are unknown, many problems can befall the unwary investigator. In this paper we explore some of these problems and discuss some solutions. Our principle example is drawn from data from students who choose to take the College Board’s Scholastic Aptitude Test (SAT). We explore methods of covariance adjustment as well as more explicitly model-based adjustment methods. Among the latter we discuss Heckman’s Selection Model, Rubin’s Mixture Model, and Tukey’s Simplified Selection Model.