This article describes the logic and procedures behind testing for factorial invariance across groups in the context of construct validation. The procedures include testing for configural, measurement, and structural invariance in the framework of multiple-group confirmatory factor analysis (CFA). The forward (sequential constraint imposition) approach to testing for factorial invariance is described and illustrated for the cases of first-order and second-order CFA models; computer codes in Mplus are provided. Computations of the Satorra—Bentler scaled chi-square difference, used in testing for factorial invariance under lack of multivariate normality, are also described. Some points of caution related to testing and interpreting measurement invariance are provided as well.
American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. ( 1999). Standards for educational and psychological testing . Washington, DC: American Educational Research Association .
Borsboom, D., Romeijn, J.W., & Wicherts, J.M. (2008). Measurement invariance versus selection invariance: Is fair selection possible? Psychological Methods, 13, 75-98.
6.
Browne, M.W., & Cudeck, R. ( 1993). Alternative ways of assessing model fit. In K. A. Bollen & J. S. Long (Eds.), Testing structural equation models (pp. 136-162). Newbury Park, CA: Sage.
7.
Byrne, B.M. ( 1988). The Self-Description Questionnaire III: Testing for equivalent factorial validity across ability. Educational and Psychological Measurement, 48, 397-406.
8.
Byrne, B.M. ( 2004). Testing for multigroup invariance using AMOS graphics: A road less traveled. Structural Equation Modeling, 11, 272-300.
9.
Byrne, B.M., Shavelson, R.J., & Muthén, B. (1989). Testing for the equivalence of factor covariance and mean structures: The issue of partial measurement invariance. Psychological Bulletin, 105, 456-466.
10.
Cattell, R.B. ( 1946). The description and measurement of personality . New York, NY: World Book.
11.
Cattell, R.B., Cattell, A.K.S., & Cattell, H.E.P. (1993). 16PF (5th ed.). Champaign, IL: Institute for Personality and Ability Testing.
Cleary, T.A. ( 1968). Test bias: Prediction of grades of Negro and White students in integrated colleges. Journal of Educational Measurement , 5, 115-124.
15.
Cliff, N. ( 1966). Orthogonal rotation to congruence. Psychometrika , 31, 33-42.
16.
Costa, P.T., Jr., & McCrae, R.R. (1990). Personality disorders and the five-factor model. Journal of Personality Disorders , 4, 362-371.
17.
Costa, P.T., Jr., & McCrae, R.R. (1991). NEO five-factor inventory (NEO-FFI), Form S (Adult). Lutz, FL: Psychological Assessment Resources.
18.
Costa, P.T., Jr., & McCrae, R.R. (2003). NEO five-factor inventory (NEO-FFI), Form S (College Age). Lutz, FL: Psychological Assessment Resources.
Curran, P.J., West, S.G., & Finch, J.F. ( 1996). The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychological Methods, 1, 16-29.
21.
Drasgow, F., & Kanfer, R. ( 1985). Equivalent of psychological measurement in heterogeneous populations. Journal of Applied Psychology, 70, 662-680.
22.
Fan, X., Thompson, B., & Wang, L. ( 1999). Effects of sample size, estimation method, and model specification on structural equation modeling fit indexes. Structural Equation Modeling, 6, 56-83.
23.
Green-Demers, I., Legault, L., Pelletier, D., & Pelletier, L.G. (2008). Factorial invariance of the academic amotivation inventory (AAI) across gender and grade in a sample of Canadian high school students. Educational and Psychological Measurement, 68, 862-880.
Horn, J.L., & McArdle, J.J. ( 1992). A practical and theoretical guide to measurement invariance in aging research. Experimental Aging Research , 18, 117-144.
26.
Hu, L.T., & Bentler, P.M. ( 1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6, 1-55.
27.
Hu, L.T., Bentler, P.M., & Kano, Y. ( 1992). Can test statistics in covariance structure analysis be trusted? Psychological Bulletin, 112, 351-362.
28.
Johnson, E.C., Meade, A.W., & DuVernet, A.M. (2009). The role of referent indicators in tests of measurement invariance. Structural Equation Modeling , 16, 642-657.
29.
Jöreskog, K.G. ( 1971). Simultaneous factor analysis in several populations . Psychometrika, 36, 409-426.
30.
Jöreskog, K.G., & Sörbom, D. (1979). Advances in factor analysis and structural equation models. Cambridge, MA: ABT Books.
Kline, R.B. ( 1998). Principles and practice of structural equation modeling . New York, NY: Guilford .
33.
Kornell, N., & Metcalfe, J. ( 2006). Study efficacy and the region of proximal learning framework. Journal of Experimental Psychology: Learning, Memory, and Cognition, 32, 609-622.
34.
Le, H., Casillas, A., Robbins, S.B., & Langley, R. ( 2005). Motivational and skills, social, self-management predictors of college outcomes: Constructing the student readiness inventory . Educational and Psychological Measurement, 65, 482-508.
35.
Levine, D.W., Kaplan, R.M., Kripke, D.F., Bowen, D.J., Naughton, M.J., & Shumaker, S.A. (2003). Factor structure and measurement invariance of the women’s health initiative insomnia rating scale, Psychological Assessment15, 123-136.
36.
Lievens, F., Anseel, F., Harris, M.M., & Eisenberg, J. ( 2007). Measurement invariance of the Pay Satisfaction Questionnaire across three countries. Educational and Psychological Measurement , 67, 1042-1051.
37.
Little, T.D. ( 1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32, 53-76.
38.
MacCallum, R.C., Wegener, D.T., Uchino, B.N., & Fabrigar, L.R. (1993). The problem of equivalent models in application of covariance structure analysis. Psychological Bulletin, 114, 185-199.
39.
Mann, H.M., Rutstein, D.W., & Hancock, G.R. ( 2009). The potential for differential findings among invariance testing strategies for multisample measured variable path models . Educational and Psychological Measurement, 69, 603-612.
40.
Marsh, H.W. ( 1986). The bias of negatively worded items in rating scales for young children: A cognitive-developmental phenomena. Developmental Psychology, 22, 37-49.
41.
Marsh, H.W., Balla, J.R., & McDonald, R.P. (1988). Goodness-of-fit indexes in confirmatory factor analysis: The effect of sample size. Psychological Bulletin, 103, 391-410.
42.
Marsh, H.W., Muthén, B., Asparouhov, T., Lüdtke, O., Robitzsch, A., Morin, A.J.S., & Trautwein, U. ( 2009). Exploratory structural equation modeling, integrating CFA and EFA: Application to student’s evaluations of university teaching . Structural Equation Modeling, 16, 439-476.
43.
Meade, A.W., & Kroustalis, C.M. (2006). Problems with item parceling for confirmatory factor analytic tests of measurement invariance. Organizational Research Methods, 9, 369-403.
44.
Meredith, W. ( 1964). Rotation to achieve factorial invariance. Psychometrika, 29, 187-206.
45.
Meredith, W. ( 1993). Measurement invariance, factor analysis and factorial invariance. Psychometrika, 58, 525-543.
46.
Meredith, W., & Teresi, J. ( 2006). An essay on measurement and factorial invariance . Medical Care, 44(Suppl. 3), S69-S77.
47.
Messick, S. ( 1989). Validity. In R. L. Linn (Ed.), Educational measurement (3rd ed., pp. 13-103). New York, NY: American Council on Education.
48.
Messick, S. ( 1995). Validity of psychological assessment: Validation of inferences from persons’ responses and performances as scientific inquiry into score meaning. American Psychologist, 50, 741-749.
49.
Millsap, R.E. ( 1997). Invariance in measurement and prediction: Their relationship in the single-factor case. Psychological Methods , 2, 248-260.
50.
Millsap, R.E. ( 1998). Group differences in regression intercepts: Implications for factorial invariance. Multivariate Behavioral Research , 33 (3), 403-424.
51.
Millsap, R.E. ( 2007). Invariance in measurement and prediction revisited . Psychometrika, 72, 461-473.
Millsap, R.E., & Kwok, O. ( 2004). Evaluating the impact of partial factorial invariance on selection in two populations. Psychological Methods, 99, 93-115.
Nasser, F., & Wisenbaker, J. (2003). A Monte Carlo study investigating the impact of item parceling on measures of fit in confirmatory factor analysis . Educational and Psychological Measurement, 63, 729-757.
56.
Raykov, T., & Marcoulides, G.A. (2006). A first course in structural equation modeling (2nd ed.). Mahwah, NJ: Lawrence Erlbaum .
57.
Rensvold, R.B., & Cheung, G.W. ( 2001). Testing for metric invariance using structural equation models: Solving the standardization problem. In C. A. Schriesheim & L. L. Neider (Eds.), Research in management: Vol. 1. Equivalence in measurement (pp. 21-50). Greenwich, CT: Information Age.
58.
Satorra, A. ( 2000). Scaled and adjusted restricted tests in multi-sample analysis of Moment structures. In R. D. H. Heijmans, D. S. G. Pollock, & A. Satorra (Eds.). Innovations in multivariate statistical analysis. A Festschrift fro Heiz Neudecker (pp. 233-247). London: Kluwer Academic Publishers.
59.
Satorra, A., & Bentler, P.M. ( 1994). Corrections to test statistics and standard errors in covariance structure analysis. In A. von Eye & C. C. Clogg (Eds.), Latent variables analysis: Applications for developmental research (pp. 399-419). Thousands Oaks, CA: Sage.
60.
Satorra, A., & Bentler, P.M. ( 1999). A scaled difference chi-square test statistic for moment structure analysis (UCLA Statistic Series, No. 260). Retrieved from http://preprints.stat.ucla.edu
61.
Satorra, A., & Bentler, P.M. ( 2001). A scaled difference chi-square test statistic for moment structure analysis. Psychometrika, 66, 507-514.
62.
Simpson, E.H. ( 1951). The interpretation of interaction in contingency tables . Journal of Royal Statistical Society, 13, 238-241.
63.
Smith, T.B., Rosenstein, I., & Granaas, M.M. ( 2001). Intake screening with the Self-Rating Depression Scale in a university counseling center. Journal of College Counseling , 4, 133-141.
64.
Spearman, C. ( 1927). The abilities of man: Their nature and measurement . New York, NY: Macmillan.
65.
Steiger, J.H. ( 1990). Structural model evaluation and modification: An interval estimation approach. Multivariate Behavioral Research , 25, 173-180.
66.
Tucker, L.R., & Lewis, C. ( 1973). A reliability coefficient for maximum likelihood factor analysis. Psychometrika, 38, 1-10.
67.
Vandenberg, R.J., & Lance, C.E. ( 2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4-70.
68.
Whitman, D.S., Van Rooy, D.L., Viswesvaran, C., & Kraus, E. ( 2009). Testing the second-order factor structure and measurement equivalence of the Wong and Law Emotional Intelligence Scale across gender and ethnicity. Educational and Psychological Measurement , 69, 1059-1074.
69.
Widaman, K.F., & Reise, S.P. ( 1997). Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. In K. J. Bryant, M. Windle , & S. G. West (Eds.), The science of prevention: Methodological advances from alcohol and substance abuse research (pp. 281-324). Washington, DC: American Psychological Association .
70.
Wolfe, E.W., & Smith, E.V. ( 2007a). Instrument development tools and activities for measure validation using Rasch models: Part I-Instrument development tools . Journal of Applied Measurement, 8, 97-123.
71.
Wolfe, E.W., & Smith, E.V. ( 2007b). Instrument development tools and activities for measure validation using Rasch models: Part II-Validation activities. Journal of Applied Measurement, 8, 204-234.
72.
Wu, A.D., & Zumbo, B.D. ( 2007). Decoding the meaning of factorial invariance and updating the practice of multi-group confirmatory factor analysis: A demonstration with TIMSS data. Practical Assessment, Research & Evaluation , 12, 1-26.
73.
Yin, P., & Fan, X.T. ( 2003). Assessing the factor structure invariance of self-concept measurement across ethnic and gender groups: Findings from a national sample . Educational and Psychological Measurement, 63, 296-318.
74.
Yuan, K.H. ( 2005). Fit statistics versus test statistics. Multivariate Behavioral Research, 40, 115-148.
75.
Yoon, M., & Millsap, R.E. ( 2007). Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Structural Equation Modeling, 14, 435-463.
76.
Zimmerman, B.J. ( 2000). Attainment of self-regulation: A social cognitive perspective . In M. Boekaerts, P. R. Pintrich, & M. Zeidner (Eds.), Handbook of self-regulation (pp. 13-39). San Diego, CA: Academic Press.
77.
Zimmerman, B., & Kitsantas, A. ( 2005). The hidden dimension of personal competence: Self-regulated learning and practice. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of competence and motivation (pp. 509-526). New York, NY : Guilford.
78.
Zumbo, B.D., Sireci, S.G., & Hambleton, R.K. (2003, April). Re-visiting exploratory methods for construct comparability: Is there something to be gained from the ways of old? Paper presented at the symposium Construct Comparability Research: Methodological Issues and Results. Organized by Kadriye Ercikan, for the National Council of Measurement in Education (NCME) meetings, Chicago, IL.
79.
Zung, W.W. ( 1965). A self-rating depression scale. Archives of General Psychiatry, 12, 63-70.
