The construct validity of three entry level personality inventories used in the UK: cautionary findings from a multiple‐inventory investigation

Abstract

This paper reports psychometric analyses into the convergent and divergent validity of three popular entry‐level measures of occupational personality in the UK and Continental Europe. A sample of 504 individuals completed all three measures: the British version of the Hogan Personality Inventory (HPI), the Occupational Personality Questionnaire (OPQ Version FS5.2), and the Business Personality Indicator (BPI). In addition, independent ratings of the conceptual loading of primary source scales onto the Five Factor Model (FFM) were obtained (n = 66). Data were used in a three‐stage analytical procedure directed at examining psychometric and construct validity. Results are reported for descriptive statistics (means, standard deviations, Cohen's d), internal consistency reliability (Cronbach's alphas), and exploratory factor analyses. Findings into the construct validity of first‐order scales (i.e. primary source scales) and second‐order scales (i.e. FFM loadings) are presented in detail, including multitrait–multimethod (MTMM) analyses of convergent and divergent validity. For some scales, the observed variability in our sample suggested significant range restriction/enhancement. It was found that scale reliabilities were generally lower than those typically reported by the test publishers, and that published factor structures for these measures could not be replicated by the authors for this sample of individuals. Further independent construct validity research into occupational personality inventories is encouraged based upon our proposed model of single‐, dual‐, and multiple‐inventory construct validation studies. Practically, our findings suggest that when IWO psychologists or personnel professionals aim to select/screen job applicants for a particular personality trait those who are selected may vary depending on (i) which personality inventory is used, (ii) the actual variability in the applicant sample tested, and (iii) reliability of the scales under consideration. As such, this study sounds a note of caution. Future research is called for to replicate these findings. Copyright © 2003 John Wiley & Sons, Ltd.

References

Anderson

(2000).The future of selection and assessment. Paper presented to the British Test User Association Conference, London.

Anderson, N. , & Herriot, P. (Eds.). (1997). International handbook of selection and assessment. Chichester: Wiley.

Anderson

Herriot

Hodgkinson

G. P.

(2001).The practitioner–researcher divide in Industrial, Work and Organizational (IWO) psychology: Where are we now, and where do we go from here? Journal of Occupational and Organizational Psychology, 74,391–411.

Anderson

West

M. A.

(1998).Measuring climate for work group innovation: Development and validation of the team climate inventory. Journal of Organizational Behavior, 19,235–258.

Barrick

M. R.

Mount

M. K.

(1991).The Big Five personality dimensions and job performance: A meta‐analysis. Personnel Psychology, 41,1–26.

Barrick

M. R.

Mount

M. K.

Judge

T. A.

(2001).Personality and performance at the beginning of the new millennium: What do we know and where do we go next? International Journal of Selection and Assessment, 9(1/2),9–30.

Bartram

Lindley

P. A.

Marshall

Foster

(1995).The recruitment and selection of young people by small businesses. Journal of Occupational and Organizational Psychology, 68,339–358.

Bond

M. H.

Nakazato

H. S.

Shiraishi

(1975).Universality and distinctiveness in dimensions of Japanese person perception. Journal of Cross‐Cultural Psychology, 6,346–355.

Borkenau

Ostendorf

(1990).Comparing exploratory and confirmatory factor analysis: A study of the 5‐factor model of personality. Personality and Individual Differences, 11,515–524.

10.

Campbell

D. T.

Fiske

D. W.

(1959).Convergent and discriminant validation by the multitrait–multimethod matrix. Psychological Bulletin, 56,81–105.

11.

Cattell

R. B.

(1966).The scree test for the number of factors. Multivariate Behavioral Research, 1,140–161.

12.

Cellar

D. F.

Miller

M. L.

Doverspike

D. D.

Klawsky

J. D.

(1996).Comparison of factor structures and criterion‐related validity coefficients for two measures of personality based on the five factor model. Journal of Applied Psychology, 81,694–704.

13.

Comrey

(1978).Common methodological problems in factor analytic studies. Journal of Clinical and Consulting Psychology, 46,648–659.

14.

Conley

J. J.

(1985). A personality theory of adulthood and aging. In R. Hogan , & W. H. Jones (Eds.), Perspectives in Personality(Vol. 1, PP.81–115).

15.

Costa

P. T.

McCrae

R. M.

(1992). Revised NEO Personality Inventory (NEO‐PI‐R) and NEO Five‐Factor Inventory (NEO‐FFI) professional manual. Odessa, FL: Psychological Assessment Resources.

16.

Costa

P. T.

McCrae

R. R.

(1988).From catalog to classification: Murray's needs and the five‐factor model. Journal of Personality and Social Psychology, 55,258–265.

17.

Digman

J. M.

(1990).Personality structure: Emergence of the Five Factor Model. Annual Review of Psychology, 41,417–440.

18.

Digman

J. M.

(1997).Higher order factors of the Big Five. Journal of Personality and Social Psychology, 73,1246–1256.

19.

Digman

J. M.

Inouye

(1986).Further specification of the five robust factors of personality. Annual Review of Personality and Social Psychology, 50,116–123.

20.

Digman

J. M.

Takemoto‐Chock

N. K.

(1981).Factors in the natural language of personality: Re‐analysis and comparison of six major studies. Multivariate Behavioral Research, 16,146–170.

21.

Feltham

Woods

(1995). Manual and User Guide for the Business Personality Indicator. Windsor: ASE.

22.

Ferguson

Payne

Anderson

(1994).Occupational personality assessment: An evaluation of the psychometric properties of the Occupational Personality Questionnaire (OPQ). Personality and Individual Differences, 17,217–225.

23.

Fiske

D. W.

(1949).Consistency of the factorial structures of personality ratings from different sources. Journal of Abnormal and Social Psychology, 44,329–344.

24.

Fleiss

J. L.

(1971).Measuring nominal scale agreement among many raters. Psychological Bulletin, 76,378–382.

25.

Furnham

(1994).A content, correlational and factor analytic study of four tolerance of ambiguity questionnaires. Personality and Individual Differences, 16,403–410.

26.

Furnham

(1996).The FIRO‐B, the Learning Style Questionnaire, and the Five Factor Model. Journal of Social Behavior and Personality, 11,285–299.

27.

Goldberg

L. R.

(1990).An alternate ‘description of personality’, the Big‐Five factor structure. Journal of Personality and Social Psychology, 59(6),1216–1229.

28.

Goldberg

L. R.

(1993).The structure of phenotypic personality traits. American Psychologist, 48,26–34.

29.

Gorsuch

(1983). Factor Analysis. London: Erlbaum.

30.

Herriot

Anderson

(1997). Selecting for change: How will personnel and selection psychology survive?In N. Anderson , & P. Herriot (Eds.), International handbook of selection and assessment(pp.1–34). Chichester: Wiley.

31.

Hodgkinson

G. P.

Payne

R. L.

(1998).Graduate selection in three European countries. Journal of Occupational and Organizational Psychology, 71,359–365.

32.

Hogan

(1995). Hogan Personality Inventory manual. Tulsa, OK: Hogan Assessment Systems.

33.

Hogan

R. T.

Roberts

B. W.

(2001). Introduction: Personality and industrial and organizational psychology. In B. W. Roberts , & R. T. Hogan (Eds.), Personality psychology in the workplace(pp.3–18). Washington, DC: American Psychological Association.

34.

Hough

L. M.

(2001). I/Owes its advances to personality. In B. W. Roberts , & R. T. Hogan (Eds.), Personality psychology in the workplace(pp.19–44). Washington, DC: American Psychological Association.

35.

Hough

L. M.

Ones

D. S.

(2001). The structure, measurement, validity, and use of personality variables in industrial, work, and organizational psychology. In N. Anderson , D. S. Ones , H. Sinangil , & C. Viswesvaran (Eds.), Handbook of industrial, work, and organizational psychology(Vol. 1, PP.233–277). London: Sage.

36.

Hough

L. M.

Oswald

F. L.

Ployhart

R. E.

(2001).Determinants, detection and amelioration of adverse impact in personnel selection procedures: Issues, evidence and lessons learned. International Journal of Selection and Assessment, 9,152–194.

37.

Kline

(1986). A handbook of test construction: Introduction to psychometric design. London: Methuen.

38.

Knowles

E. S.

(1988).Item context effects on personality scales: Measuring changes the measure. Journal of Personality and Social Psychology, 55,312–320.

39.

Lorr

Youniss

R. P.

(1973).An inventory of interpersonal style. Journal of Personality Assessment, 37,165–173.

40.

Matthews

Stanton

Graham

N. C.

Brimelow

(1990).A factor analysis of the scales of the Occupational Personality Questionnaire. Personality & Individual Differences, 11(6),591–596.

41.

McCrae

R. R.

(2001).Trait psychology and culture: Exploring intercultural comparisons. Journal of Personality, 69,819–846.

42.

McCrae

R. R.

Costa

P. T.

(1987).Validation of the five‐factor model of personality across instruments and observers. Journal of Personality and Social Psychology, 52,81–90.

43.

McCrae

R. R.

Costa

P. T.

(1997).Personality trait structure as a human universal. American Psychologist, 52(5),509–516.

44.

McCrae

R. R.

John

O. P.

(1992).An introduction to the five‐factor model and its applications. Journal of Personality, 60,175–216.

45.

Norman

W. T.

(1963).Toward an adequate taxonomy of personality attributes: Replicated factor structure in peer nomination personality ratings. Journal of Abnormal and Social Psychology, 66,574–582.

46.

Norman

W. T.

Goldberg

L. R.

(1966).Raters, ratees, and randomness in personality structure. Journal of Personality and Social Psychology, 4,681–691.

47.

Ones

D. S.

(1993).The Construct Validity Evidence for Integrity Tests, copyrighted doctoral dissertation, Iowa City, Iowa: University of Iowa.

48.

Ones

D. S.

Anderson

(2002).Gender and ethnic group differences on personality scales in selection: Some British data. Journal of Occupational and Organizational Psychology, 75(3),255–276.

49.

Ones

D. S.

Schmidt

F. L.

Viswesvaran

(1995, August). Convergence among three personality inventories measuring the Big Five. Paper presented at the 103rd annual meeting of the American Psychological Association, New York.

50.

Ones

D. S.

Viswesvaran

(1996).Bandwidth–fidelity dilemma in personality measurement for personnel selection. Journal of Organizational Behavior, 17,209–226.

51.

Ones

D. S.

Viswesvaran

(1998).Gender, age and race differences on overt integrity tests: Analyses across four large‐scale applicant data sets. Journal of Applied Psychology, 83,35–42.

52.

Ones

D. S.

Viswesvaran

(2001).Integrity tests and other criterion‐focused occupational personality scales (COPS) used in personnel selection. International Journal of Selection and Assessment, 9,31–39.

53.

Ones

D. S.

Viswesvaran

Schmidt

F. L.

(1993).Comprehensive meta‐analysis of integrity test validities: Findings and implications for personnel selection and theories of job performance. Journal of Applied Psychology (Monograph), 78,679–703.

54.

Robertson

I. T.

Kinder

(1993).Personality and job competences: The criterion‐related validity of some personality variables. Journal of Occupational and Organizational Psychology, 66,225–244.

55.

Rothmann

Meiring

Barrick

M. R.

(2002).Predicting job performance using personality measures in South Africa: A meta‐analytic study. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Toronto.

56.

Ryan

A. M.

McFarland

Baron

Page

(1999).An international look at selection practices: Nation and culture as explanations for variability in practice. Personnel Psychology, 52,359–392.

57.

Salgado

J. F.

(1996).Personality and job competences: A comment on the Robertson and Kinder (1993) study. Journal of Occupational and Organizational Psychology, 69(4),346–351.

58.

Salgado

J. F.

(1997).The five factor model of personality and job performance in the European community. Journal of Applied Psychology, 82,30–43.

59.

Salgado

J. F.

(2002).The Big Five personality dimensions and counterproductive behaviors. International Journal of Selection and Assessment, 10,117–125.

60.

Salgado

J. F.

Anderson

(2002).Cognitive and GMA testing in the European Community: Issues and evidence. Human Performance, 15,75–96.

61.

Salgado

J. F.

Viswesvaran

Ones

D. S.

(2001). Predictors used for personnel selection: An overview of constructs, methods and techniques. In N. Anderson , D. S. Ones , H. Sinangil , & C. Viswesvaran (Eds.), Handbook of industrial, work and organizational psychology(Vol. I). (pp.165–199). London: Sage.

62.

Shackleton

V. J.

Newell

(1994).European management selection methods: A comparison of five countries. International Journal of Selection and Assessment, 2,91–102.

63.

SHL. (1999). OPQ32 Manual and User's Guide. Surrey: SHL.

64.

Shrout

P. E.

Fleiss

J. L.

(1979).Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86,420–428.

65.

Thompson

Borrello

G. M.

(1986).Second‐order factor structure of the MBTI: A construct validity assessment. Measurement and Evaluation in Counselling and Development,January,148–153.

66.

Watson

(1989).Strangers' ratings of the five robust personality factors: Evidence of a surprising convergence with self‐report. Journal of Personality and Social Psychology, 57,120–128.

67.

Yang

Bond

M. H.

(1990).Exploring implicit personality theories with indigenous or imported constructs: The Chinese case. Journal of Personality and Social Psychology, 58,1087–1095.

68.

Yoo

T. Y.

Min

B. M.

(2002).A meta‐analysis of the Big Five and performance in Korea. Paper presented at the annual conference of the Society for Industrial and Organizational Psychology, Toronto.