Social survey data are often collected using complex sample designs that include unequal selection probabilities, stratification, and clustering; and this departure from simple random sampling assumptions requires special methods for variance estimation and inferential analysis. This article reviews the issues and techniques for analysis of complex survey data; and the application of special techniques is illustrated, based on various complex survey data. Discussion centers on methodological and some practical issues in data analysis and the need for proper analysis of complex survey data.
References
1.
ALTHAM, P.M.E.
(1976) “Discrete variable analysis for individuals grouped into families.”Biometrika63: 263-269.
2.
BEAN, J. A.
(1970) Estimation and Sampling Variance in the Health Interview Survey. Vital and Health Statistics Series 2, No. 38, NCHS. Rockville, MD: National Center for Health Statistics.
3.
BEAN, J. A.
(1975) Distribution and Properties of Variance Estimators for Complex Multistage Probability Samples: An Empirical Distribution. Vital and Health Statistics, Series 2, No. 65,National Center for Health Statistics.
4.
BEDRICK, E. J.
(1983) “Adjusted chi-square tests for cross-classified tables of survey data.”Biometrika70: 591-595.
5.
BRIER, S. S.
(1980) “Analysis of contingency tables under cluster sampling.”Biometrika67: 591-596.
6.
BRILLINGER, D. R.
(1964) “The asymptotic behavior of Tukey's general method of setting approximate confidence limits (the jackknife) when applied to maximum likelihood estimates.”Rev. Int. Stat. Inst.3: 202-206.
7.
BURT, V. L.
and S. B. COHEN (1984) “A comparison of methods to approximate standard errors for complex survey data.”Rev. of Public Data Use12: 159-168.
8.
CHAI, J. J.
(1979) “Correlated measurement errors and least squares estimation of the regression coefficient.”J. of the Amer. Stat. Assn.66: 478-483.
9.
COCHRAN, W.G.
(1968) “Errors of measurement in statistics.”Technometrics10: 637-666.
10.
COCHRAN, W.G.
(1977) Sampling Techniques. New York: John Wiley.
11.
COHEN, J. E.
(1976) “The distribution of the chi-squared statistic under clustered sampling from contingency tables.”J. of the Amer. Stat. Assn.71: 665-670.
12.
DAVIS, M. A.
, E. RANDALL, R. N. FORTHOFER, E. S. LEE, and S. MORGAN (1985) “Living arrangements and dietary patterns of older adults in the United States.”J. of Gerontology40: 434-442.
13.
DeMETS, D.
and M. HALPERIN (1977) “Estimation of a simple regression coefficient in samples arising from a subsampling procedure.”Biometrics33: 47-56.
14.
DEMING, W. E.
(1956) “On simplification of sampling design through replication with equal probabilities and without stages.”J. of the Amer. Stat. Assn.51: 24-53.
15.
DEMING, W. E.
(1960) Sample Design in Business Research. New York: John Wiley.
16.
DuMOUCHEL, W. H.
and G. J. DUNCAN (1983) “Using sample survey weights in multiple regression analysis of stratified samples.”J. of the Amer. Stat. Assn.78: 535-543.
17.
FARKAS, G.
(1974) “Specification, residuals and contextual effects.”Soc. Meth. Research2: 333-365.
18.
FAY, R. E.
(1982) “Contingency table analysis for complex designs: CPLX,” in Proceedings of the Section on Survey Research Methods, American Statistical Association: 44-53.
19.
FAY, R. E.(1983) “CPLX—contingency table analysis for complex sample designs, program documentation.” Report U.S. Bureau of the Census. (unpublished)
20.
FAY, R. E.
(1985) “A jackknife chi-square test for complex samples.”J. of the Amer. Stat. Assn.80: 148-157.
21.
FELLIGI, I. P.
(1980) “Approximate tests of independence and goodness of fit based on stratified multi-stage samples.”J. of the Amer. Stat. Assn.75: 261-268.
22.
FORTHOFER, R. N.
and R. G. LEHNEN (1981) Public Program Analysis: A Categorical Data Approach. Belmont, CA: Lifetime Learning Publications.
23.
FRANKEL, M. R.
(1971) Inference from Survey Samples. Ann Arbor: Institute for Social Research, Univ. of Michigan.
24.
FRANKEL, M. R.
(1983) “Sampling theory,” pp. 21-67 in P. H. Rossiet al. (eds.) Handbook of Survey Research. New York: Academic Press.
25.
FREEMAN, D. H.
, J. L. FREEMAN, D. B. BROCK, and G. G. KOCH (1976) “Strategies in the multivariate analysis of data from complex surveys II: an application to the United States National Health Interview Survey.”Int. Stat. Rev.44: 317-330.
26.
FULLER, W. A.
(1975) “Regression analysis for sample survey.”Sankhya (C)37: 117-132.
27.
GOODMAN, L. A.
(1972a) “A general model for the analysis of survey.”Amer. J. of Sociology77: 1035-1086.
28.
GOODMAN, L. A.
(1972b) “A modified multiple regression approach to the analysis of dichotomous variables.”Amer. Soc. Rev.37: 28-45.
29.
GOODMAN, L. A.
(1979) “A brief guide to the causal analysis of data from surveys.”Amer. J. of Sociology84: 1078-1095.
30.
GRIZZLE, J. E.
, C. F. STARMER, and G. G. KOCH (1969) “Analysis of categorical data by linear models.”Biometrics25: 489-504.
31.
GURNEY, M.
and R. S. JEWETT (1975) “Constructing orthogonal replications for variance estimation.”J. of the Amer. Stat. Assn.70: 819-821.
32.
HADDEN, W. C.
(1980) Basic Data on Health Care Needs of Adults Ages 25-74 Years, United States, 1971-75, Vital and Health Statistics, Series 11, No. 218, National Center for Health Statistics.
33.
HANSEN, M. H.
, W. G. MADOW, and B. J. TEPPING (1983) “An evaluation of model-dependent and probability-sampling inferences in sample surveys.”J. of the Amer. Stat. Assn.78: 776-807.
34.
HAUSER, R. M.
(1974) “Contextual analysis revisited.”Soc. Meth. Research2: 365-375.
35.
HIDIROGLOU, M. A.
, W. A. FULLER, and R. D. HICKMAN (1980) SUPER CARP. Survey Section, Statistical Laboratory, Iowa State Univ.
36.
HOLT, D.
and A. J. SCOTT (1981) “Regression analysis using survey data.”The Statistician30: 169-178.
37.
HOLT, D.A. J. SCOTT
and P. O. EWINGS (1980) “Chi-squared tests with survey data.”J.R. Statist. Soc. (A)143: 302-320.
38.
HOLT, D.
, T.M.F. SMITH, and P. D. WINTER (1980) “Regression analysis of data from complex surveys.”J.R. Statist. Soc. (A)143: 474-487.
39.
HOLT, M. M. (1977) “SURREGR: standard errors of regression coefficients from sample survey data.” Report from Research Triangle Institute, Research Triangle Park, N.C., revised by B. V. Shah in 1982. (unpublished)
40.
HORVITZ, D. G.
and D. J. THOMPSON (1952) “A generalization of sampling without replacement from a finite population.”J. of the Amer. Stat. Assn.47: 663-685.
41.
JENNRICH, R. I.
and P. F. SAMPSON (1976) Newton-Raphson and related algorithms for maximum likelihood variance component estimation. Technometrics18: 11-15.
42.
KALTON, G.
(1983) Introduction to Survey Sampling. Beverly Hills, CA: Sage.
43.
KENDALL, P. A.
and P. F. LAZARSFELD (1950) “Problems of survey analysis,” pp. 133-196 in R. K. Merton and P. F. Lazarsfeld (eds.) Continuities in Social Research: Studies in the Scope and Method of “The American Soldier.”New York: Free Press.
44.
KERR, G. R.
, E. S. LEE, M.K.M. LAM, R. J. LORIMOR, E. RANDALL, and R. N. FORTHOFER (1982) “Relationships between dietary and biochemical measures of nutritional status in NHANES I data.”J. of Clinical Nutrition35: 294-308.
45.
KEYFITZ, N.
(1957) “Estimates of sampling variance where two units are selected from each stratum.”J. of the Amer. Stat. Assn.52: 503-510.
46.
KISH, L.
(1949) “A procedure for objective respondent selection within the household.”J. of the Amer. Stat. Assn.44: 380-387.
47.
KISH, L.
(1965) Survey Sampling. New York: John Wiley.
48.
KISH, L.
and M. R. FRANKEL (1970) Balanced repeated replication for standard errors.”J. of the Amer. Stat. Assn.65: 1071-1094.
49.
KISH, L.
and M. R. FRANKEL(1974) “Inferences from complex samples.”J.R. Statist. Soc. (B)36: 1-37.
50.
KISH, L.M. R. FRANKEL
and N. V. ECK (1972) SEPP: Sampling Error Program Package. Ann Arbor: Univ. of Michigan Press.
51.
KOCH, G. G.
, D. H. FREEMAN, and J. L. FREEMAN (1975) “Strategies in the multivariate analysis of data from complex surveys.”Int. Stat. Rev.43: 59-78.
52.
KOCH, G. G.
and S. LEMESHOW (1972) “An application of multivariate analysis to complex sample survey data.”J. of the Amer. Stat. Assn.67: 780-782.
53.
KONIJN, H.
(1962) “Regression analysis in sample surveys.”J. of the Amer. Stat. Assn.57: 590-605.
54.
KREWSKI, D.
and J.N.K. RAO (1981) “Inference from stratified samples: properties of the linearization, jackknife and balanced repeated replication methods.”Annals of Statistics9: 1010-1019.
55.
LANDIS, J. R.et al.
(1976) “A computer program for the generalized chi-square analysis of categorical data using weighted least squares (GENCAT).”Computer Programs in Biomedicine6: 196-231.
56.
LANDIS, J. R.
, J. M. LEPKOWSKI, S. A. EKLUND, and S. A. STEHOUWER (1982) A Statistical Methodology for Analyzing Data from a Complex Sample Survey: The First National Health and Nutrition Examination Survey. Vital and Health Statistics Series 2, No. 92, National Center for Health Statistics.
57.
LEE, E. S.
(1986) “An evaluation of pseudodesign-based sampling errors in the St. Louis epidemiologic Catchment Area Survey.” Prepared for Division of Clinical Research, National Institute of Mental Health. (unpublished)
58.
LEE, E. S.
and R. N. FORTHOFER (1983) Evaluation of the DHHS Proposed Diagnosis Related Groups (DRGs) for Mental Health. Rockville, MD: National Institute of Mental Health.
59.
LEE, E. S.R. N. FORTHOFER
and C. E. HOLZER (1984) “Estimating standard errors from complex sample surveys: an evaluation of pseudoreplication based on sequentially partitioned poststrata.” Prepared for Division of Biometry and Epidemiology, National Institute of Mental Health. (unpublished)
60.
LEMESHOW, S.
and P. S. LEVY (1979) “Estimating the variance of ratio estimates in complex sample surveys with two primary sampling units per stratum: a comparison of balanced replication and jackknife techniques.”J. Stat. Comput. Simul.8: 191-205.
61.
LITTLE, R.J.A.
(1983) “Estimating a finite population mean from unequal probability samples.”j. of the Amer. Stat. Assn.78: 596-604.
62.
MALEC, D.
and J. SEDRANSK (1985) “Bayesian inference for finite population parameters in multistage cluster sampling.”J. of the Amer. Stat. Assn.80: 897-902.
63.
McCARTHY, P. J.
(1966) Replication: An Approach to the Analysis of Data from Complex Surveys. Vital and Health Statistics Report, Series 2, No. 14, National Center for Health Statistics.
64.
McCARTHY, P. J.
(1969) Pseudoreplication: Further Evaluations and Application of Balanced Half-Sample Technique. Vital and Health Statistics Series 2, No. 31, National Center for Health Statistics.
65.
MORGANSTEIN, D.
, R. H. HANSEN, and G. BINZER (1982) “Automation of Sampling Error Computations for Complex Estimators.”Proceedings of the Section on Survey Research Methods, American Statistical Association.
66.
MOSTELLER, F.
and J. W. TUKEY (1968) “Data analysis, including statistics,” pp. 80-203 in G. Lindsey and E. Aronson (eds.) The Handbook of Social Psychology, Vol. 2. Reading, MA: Addison-Wesley.
67.
NATHAN, G.
and D. HOLT (1980) “The effect of survey design on regression analysis.”J.R. Statist. Soc. (B)42: 377-386.
68.
PFEFFERMANN, D.
and G. NATHAN (1981) “Regression analysis of data from a cluster sample.”J. of the Amer. Stat. Assn.76: 681-689.
69.
PLACKETT, R. L.
and P. J. BURMAN (1946) “The design of optimum multifactorial experiments.”Biometrika33: 305-325.
70.
PORTER, R. M.
(1973) “On the use of survey sample weights in the linear model.”Annal Econ. Soc. Meas.2: 141-158.
71.
PROCTOR, C. H.
(1976) “Testing for contextual effect.”Proc. Am. Statist. Assn. Soc. Statist. Sec., 695-699.
72.
RAO, J.N.K.
and A. J. SCOTT (1981) “The analysis of categorical data from complex sample surveys: chi-squared tests for goodness of fit and independence in two-way tables.”J. of the Amer. Stat. Assn.76: 221-230.
73.
RAO, J.N.K.
and A. J. SCOTT(1984) “On chi-squared tests for multiway contingency tables with cell proportions estimated from survey data.”Annals of Statistics12: 46-60.
74.
REISS, A. J., Jr.
(1986) “Measuring social change,” pp. 36-72 in N. J. Smelser and D. R. Gerstein (eds.) Behavioral and Social Science: Fifty Years of Discovery. Washington, DC: National Academy Press.
75.
ROSEN, A.
and D. CHRISTIANSEN (1984) Regression Analysis of Complex Samples Using Weighted Least Squares Statistical Software. Lipid Research Clinics Program, Department of Biostatistics, University of North Carolina, Chapel Hill.
76.
ROYALL, R. M.
(1970) “On finite population sampling theory under certain linear regression models.”Biometrika57: 377-387.
77.
ROYALL, R. M.
(1976) “The linear least squares prediction approach to two-stage sampling.”J. of the Amer. Stat. Assn.71: 857-864.
78.
SÄRNADAL, C. E.
(1978) “Design-based and model-based inference in survey sampling.”Scandinavian J. of Statistics5: 25-52.
79.
SÄRNADAL, C. E.
(1980) “On π-inverse weighting versus best linear unbiased weighting in probability sampling.”Biometrika67: 639-650.
80.
SEARLE, S. R.
(1971) Linear Models. New York: John Wiley.
81.
SHAH, B. V.
(1978) “Variance estimates for complex statistics from multi-stage sample surveys,” pp. 24-35 in K. N. Namboodiri (ed.) Survey Sampling and Measurement. New York: Academic Press.
82.
SHAH, B. V.(1980) “SESUDAAN: standard error program for computing standardized rates from sample survey data.” Report from the Research Triangle Institute, Research Triangle Park, NC. (unpublished)
83.
SHAH, B. V.M. M. HOLT
, and R. E. FOLSOM (1977) “Inference about regression models from sample survey data.”Bull. of the Int. Stat. Inst.47: 43-57.
84.
SMITH, T.M.F.
(1976) “The foundations of survey sampling: a review.”J.R. Statist. Soc. (A)139: 183-204.
85.
SMITH, T.M.F.
(1983) “On the validity of inferences from non-random samples.”J R. Statist. Soc. (A)146: 394-403.
86.
STANEK, E. J.
and S. LEMESHOW (1977) “The behavior of balanced half-sample variance estimate for linear and combined ratio estimates when strata are paired to form pseudo strata.”Amer. Stat. Assn. Proc. Soc. Statist. Sec., 837-842.
87.
STEPHAN, F. F.
(1936) “Practical problems of sampling procedure.”Amer. Soc. Rev.1: 569-580.
88.
STEPHAN, F. F.
(1948) “History of the uses of modern sampling procedures.”J. of the Amer. Stat. Assn.43 (241): 12-39.
89.
SUDMAN, S.
(1976) Applied Sampling. New York: Academic Press.
90.
SWAFFORD, M.
(1980) “Three parametric techniques for contingency table analysis: nontechnical commentary.”Amer. Soc. Rev.45: 604-690.
91.
TAUBE, C. A.
, E. S. LEE, and R. N. FORTHOFER (1984) “DRGs in psychiatry: an empirical evaluation.”Medical Care22: 597-610.
92.
TAYLOR, C. C.
(1919) The Social Survey, Its History and Methods. Social Science Series 3, The University of Missouri Bulletin, Vol. 20, No. 28, Columbia, MO: Univ. of Missouri.
93.
TOMBERLIN, T. J.
(1980) “A model-based approach to the analysis of contingency tables of data from complex samples.”Amer. Stat. Assn. Proc. Sur. Res. Sec., 230-234.
94.
University of Michigan, Survey Research Center Computer Support Group
(1979) OSIRIS IV User's Manual, Institute for Social Research.
95.
VALLIANT, R.
(1985) “Nonlinear prediction theory and the estimation of proportions in a finite population.”J. of the Amer. Stat. Assn.80: 631-641.
96.
WOLTER, K. M.
(1985) Introduction to Variance Estimation. New York: Springer-Verlag.
97.
WOODRUFF, R. S.
(1971) “A simple method for approximating the variance of a complicated estimate.”J. of the Amer. Stat. Assn.66: 411-414.
98.
WOODRUFF, R. S.
and B. D. CAUSEY (1976) “Computerized method for approximating the variance of a complicated estimate.”J. of the Amer. Stat. Assn.71: 315-321.
