What Do Interest Inventories Measure? The Convergence and Content Validity of Four RIASEC Inventories

Abstract

Despite the widespread use of RIASEC interest inventories, little is known about whether these inventories actually measure the same core constructs and provide similar career recommendations to individuals. This study investigates the construct validity among four major interest inventories—the Self-Directed Search (SDS), O*NET Interest Profiler (IP), ACT Interest Inventory (UNIACT), and Strong Interest Inventory (SII). Results showed that RIASEC interest scores from the four inventories were highly correlated, but the measures often gave respondents different high-point codes. Item content analysis revealed that the basic interests reflected in each RIASEC scale both overlapped and diverged across inventories, providing an explanation for why RIASEC inventories are not interchangeable. We integrate findings across our analyses to offer cautionary notes for choosing among established RIASEC inventories and interpreting interest results. Furthermore, we also provide recommendations for constructing the next generation of basic interest inventories.

Keywords

vocational interests measurement multitrait multimethod Holland’s RIASEC basic interests convergent validity content validity

Interest inventories are widely used to help people make decisions in educational and occupational settings (Hanna & Rounds, 2020). Since the 1970s, Holland’s (1973, 1985, 1997) RIASEC model has been used to assess and organize interest types in vocational interest inventories. The RIASEC model provides a theoretical framework that helps individuals understand their interests in six general domains: Realistic (R), Investigative (I), Artistic (A), Social (S), Enterprising (E), and Conventional (C). Despite the popularity of Holland’s RIASEC model, few studies have compared what is being assessed by distinct inventories adopting the model. As a result, little is known about the convergent validity of RIASEC scores from different interest inventories.

Understanding the convergence and content coverage among interest inventories is critical for both practice and research. In practice, individuals (and career counselors) often use RIASEC interest profiles to help focus career and educational exploration. Occupations are typically considered good fits when they have similar patterns of RIASEC scores to individuals. To work effectively on a large scale, this method of career matching requires that interest inventories bear similar reflections of the RIASEC constructs. If interest inventories do not converge on their representation of Holland’s model, an individual could receive different RIASEC profiles from different inventories, and would therefore be pointed towards divergent careers. Convergent validity is also critical for research purposes. The RIASEC types are complex, multidimensional constructs that each cover a range of basic interests (Su et al., 2019). For cumulative knowledge to progress about the structure, development, and correlates of vocational interests, it is essential that researchers are aware of how item content both overlaps and diverges across different measures.

In this study, we investigate evidence for convergence and content validity among four of the most widely used interest inventories. We use both statistical and content-based approaches to examine: (a) the Self-Directed Search (SDS; Holland et al., 1994), (b) the O*NET Interest Profiler Short Form (IP; Rounds et al., 2021), (c) the ACT Interest Inventory (UNIACT-R; ACT, 1995), and (d) the Strong Interest Inventory (SII; Harmon et al., 1994). Overall, our study has three major aims. First, we examine interest score convergence by fitting Confirmatory Factor Analysis (CFA) models to the multitrait-multimethod (MTMM) matrix to reveal the relative influence of trait and method factors in influencing interest scores. Second, we build on the CFA-MTMM approach to assess convergence in terms of RIASEC high-point code agreements, a central issue for career counseling. Third, we systematically examine item content coverage using basic interests, which are narrower interest categories that underly the RIASEC themes (Su et al., 2019). Each approach provides distinct contributions to better understanding interest measurement—the first two quantitative approaches update current understanding of the convergent validity among RIASEC measures, whereas the content analysis provides critical insight into what is actually measured by different inventories. Based on our results, we offer guidance to both practitioners and researchers for choosing among existing RIASEC measures. In addition, we put forward important implications for developing future interest inventories with a basic interest structure.

Holland’s RIASEC Model

Holland’s RIASEC model provides a unified classification system for vocational interests and occupational environments (Campbell & Borgen, 1999; Campbell & Holland, 1972). Each of the RIASEC categories is a constellation of interests, preferred activities, beliefs, abilities, values, and characteristics, and can be used to describe both an individual’s interests and what their work environment supplies. Realistic (R) interests correspond to working with hands or tools (e.g., woodworking; mechanics). Investigative (I) interests are involved in scholarly or scientific activities and occupations (e.g., conduct physics research; psychologist). Artistic (A) interests correspond to being creative and expressive (e.g., paint a portrait; dancer). Social (S) interests are demonstrated in activities and occupations that involve helping or nurturing (e.g., nursing; elementary school teacher). Enterprising (E) interests entail business-related or influencing activities and occupations (e.g., open one’s own store; factory manager). Lastly, Conventional (C) interests are involved in work that is ordered and systematic (e.g., track shipping records; accountant).

Holland proposes that people tend to seek out work environments that supply interests based on their rank-ordered preferences for RIASEC types (Holland, 1985; 1997). As a career counseling tool, Holland’s RIASEC model is widely employed in career guidance, organizations (Su & Nye, 2017), and educational settings (Prediger & Swaney, 2004). The RIASEC model is also the dominant measurement framework for vocational interest research. For example, RIASEC interests predict people’s life goals (Stoll et al., 2020), influence academic and career choices (Hanna & Rounds, 2020; Usslepp et al., 2020), and predict job performance, satisfaction, and career success (Hoff et al., 2020; 2022; Nye et al., 2012; 2017; Rounds & Su, 2014; Van Iddekinge et al., 2011). Holland’s model also serves as the predominant framework for integrating interests with other individual difference domains, such as personality (Mount et al., 2005) and cognitive ability (Passler et al., 2015).

Assessing Convergence Among RIASEC Interest Inventories

With the widening application of Holland’s model, numerous measures have been developed to assess the RIASEC categories. However, a key question remains: to what extent do different interest inventories measure the same core content and provide people with similar results and career options? Or in other words, to what extent are RIASEC inventories interchangeable? Our study uses Holland’s theoretical model, and corresponding basic interests (Su et al., 2019), to investigate evidence for construct validity in three complementary ways. The first approach is based on correlations of RIASEC scores from different inventories using a multitrait-multimethod (MTMM) correlation matrix. Although other methods have been used to quantify convergent validity (e.g., regression coefficients, alerting correlation, and profile analysis), they are often built on correlation coefficients and therefore, would provide similar information to our first approach (Furr, & Bacharach, 2006). In addition, statistical models based on a MTMM correlation matrix can more effectively partition the separate influences of interest traits and inventory methods on RIASEC scores (Eid et al., 2006). The second approach uses high-point code convergence to examine a critical issue for practice: whether different inventories assign similar high-point codes (i.e., the most highly ranked interest scale) to individuals.

Together, these first two approaches offer valuable quantitative information on the degree of convergence among RIASEC measures. Yet neither of these two methods provides information about why interest scores do or do not converge from different measures. To understand the why, it is necessary to carefully examine the content coverage of each RIASEC scale within different measures. Hence, in a third approach, we adapt methods for establishing content validity and ask subject experts to examine item content similarities and differences using basic interests. We next discuss each approach in greater detail and propose hypotheses and research questions.

Correlation-Based Convergence

Most previous research on the convergent validity of interest inventories has used correlation-based approaches with only two measures. A summary of past correlational studies is reported in Table 1. The majority of these studies were found in inventory manuals, where convergent validity information is provided to show that a new interest measure is assessing the same intended constructs as an established measure. In particular, strong correlations among same-named scales across measures (e.g., Realistic-Realistic) indicate that participants who scored high on a RIASEC scale from one measure also score high on the same scale from another measure. Overall, past research indicates that cross-correlations among same- or similar-named interest scales are relatively high (r = .36–.81; r_median = .62; Table 1).

Table 1.

Cross-Correlation Between Similar or Same-Named Interest Scales.

Inventory 1	Inventory 2	Cross Correlation	N	Gender	Sample Description	Source
IP long form	Interest finder	.79	1061	56% F	Representative sample from four U.S. States	Rounds et al., 1999
IP short	Interest finder	.77	1061	56% F	Representative sample from four U.S. States	Rounds et al., 2010
IP	SII	.64	313	65% F	College students	Armstrong et al., 2008
ACT-VIP	SVIB	.81	62	100% M	Male college seniors	Hanson, 1974
ACT-VIP	SVIB	.78	83	100% F	Female college seniors	Hanson, 1974
ACT-VIP	SVIB	.72	126	100% F	Female college freshman	Hanson et al., 1974
ACT-VIP	SCII	.68	91	60% F	High school seniors	Fabry et al., 1978
ACT-VIP	VPI	.53	338	43% F	11th graders	Lamb & Prediger, 1981
ACT-VIP	KGIS	.44^a	243		9th graders	Lamb & Prediger, 1981
ACT-VIP	OVIS	.52^a	271		9th graders	Lamb & Prediger, 1981
VPI	SVIB	.62	93		College students	Blakeney et al., 1972
VPI	SVIB	.72	394	25% F	College students & job candidates, many in realistic industries	Lowman et al., 2003
VPI	SDS	.71	319
SVIB	SDS	.76	283
SII	SDS	.72	118	76% F	Career counseling practitioners and professionals who attended the society for vocational psychology conference	Savickas et al., 2002
SII	UNIACT	.65
SII	CISS	.62^a
SII	KOIS	.42^a
SDS	UNIACT	.59
SDS	CISS	.54^a
SDS	KOIS	.42^a
UNIACT	CISS	.46^a
UNIACT	KOIS	.36^a
CISS	KOIS	.37^a

Note. IP = Interest Profiler; SII = Strong Interest Inventory; ACT–VIP = ACT Vocational Interest Profile (1971–1974); SVIB = Strong Vocational Interest Bank; SCII = Strong-Campbell Interest Inventory; VPI = Vocational Preference Inventory; KGIS = Kuder General Interest Survey; OVIS = Ohio Vocational Interest Survey; CISS = Campbell Interest and Skills Survey; KOIS = Kuder Occupational Interest Survey; UNIACT-R = Revised Unisex ACT Interest Inventory (1995).

^aMean within corresponding scales are calculated first before averaging across scales.

Only a few studies have assessed correlations among more than two interest inventories. Lowman et al. (2003) compared interest scores from a relatively diverse group of participants, including college students and job candidates, and they reported moderately high correlations (r = .61–.86, r_mean = .73) among corresponding RIASEC scales measured by the Self-Directed Search, Vocational Preference Inventory, and Strong Vocational Interest Blank. In addition, Savickas et al. (2002) reported moderate correlations (r = .36–.72, r_median = .59) among similar and same-named scales from five commonly used interest inventories that were developed by distinct research groups. Together, these two studies suggest fair convergent validity among interest measures, but unique variances still exist. However, these studies only focused on bivariate correlations and were not able to quantify the amount of shared variance among more than two measures. Thus, further research is needed to examine convergent validity among interest inventories using more advanced methods that provide a more holistic view.

In this study, we use the recommended set of Confirmatory Factor Analysis (CFA) models to assess convergent and discriminant validity evidence from multitrait-multimethod (MTMM) matrices (Campbell & Fiske, 1959). These models provide a method for estimating the independent effects (factor loadings) of method and trait latent factors on participant scores (Eid et al., 2006). Correlation-based convergence is assessed in these CFA models by comparing the variance explained by interest traits versus inventory methods. If interest traits explain considerably more variance in RIASEC scores compared to inventory methods, this provides evidence for convergence. Conversely, if inventory methods explain more variance in RIASEC scores than the traits, this suggests a lack of convergence (Widaman, 1985). Thus, with the first set of analyses, we test the following hypotheses:

Hypothesis 1a. In comparison to inventory method factors, interest trait factors will explain more variance in observed RIASEC scores.

Hypothesis 1b. Inventory method factors will explain a meaningful amount of unique variance in observed RIASEC scores above and beyond interest trait factors.

High-Point Code Convergence

High-point codes are a second important way of examining convergent validity among interest inventories. High-point codes, which reflect an individual’s strongest interest area, are widely used in both practice and research. In career guidance settings, high-point codes are often used to focus individuals’ career exploration efforts (Hanna & Rounds, 2020). High-point codes are readily interpretable for individuals across education levels, and they allow for direct linkage into occupations (Harmon et al., 1994). In research, high-point interests codes have been used to predict career choice (Hanna & Rounds, 2020) and outcomes within work environments, such as job satisfaction and performance (Hoff et al., 2020; Nye et al., 2017).

Only a few studies have examined the convergence of high-point codes from different inventories. We identified three studies that found that RIASEC high-point codes were not highly consistent across interest inventories (Harmon & Zytowski, 1980; Lowman et al., 2003; Savickas & Taber, 2006). In particular, Savickas and Taber (2006) showed that, between pairs of five interest inventories, exact three-letter high-point code matches only occurred at the frequency of 3–15%. Nevertheless, the authors reported concerns about the generalizability of these results since the participants primarily consisted of career counselors and researchers attending the Society for Vocational Psychology conference. Thus, it is important to revisit and further investigate high-point code convergence with a sample more representative of typical interest inventory users (Lowman, 2022).

Here, we examine the convergence of high-point codes by calculating the percentage of participants who obtain the same high-point code on two inventories (Rounds et al., 2021). High percentages of high-point code matches indicate that the inventories are likely to provide similar career exploration information to individuals. Although there is no widely accepted benchmark for the percentages that would support convergence, guidance is available from past research on matched high-point codes from different versions of the same inventory. Specifically, the hit rate between the 30- and 60-item O*NET Interest Profiler is 69.2%, and the hit rate between the 60- and 180-item versions is 78.4% (Rounds et al., 2021). Comparatively, we expect hit rates between different inventories to be lower. Therefore, we developed the following hypothesis:

In addition to the ceiling effect that can be inferred from past literature, we can also hypothesize about the lower bound of RIASEC high-point code convergence. Hanna and Rounds (2020) reported that on average, interest high-point codes have a 50.8% hit rate for predicting career choices. This meta-analytic hit rate denotes the criterion-related validity of RIASEC measures at large. In general, we expect the reliability of RIASEC high-point codes to exceed their criterion validity. Thus, we developed the following hypothesis:

Hypothesis 2: We expect that 50–70% of participants will receive the same RIASEC high-point codes from two different interest inventories.

Item Content-Based Convergence

A third way to assess validity is to systematically examine item content. Correlation-based convergent validity does not guarantee content validity (Dixon & Johnston, 2019), which means that reasonable score correlations between two measures do not necessarily indicate that they measure the intended underlying constructs. Each RIASEC category contains a wide range of work activities and environments that measure distinct basic interests (Su et al., 2019). If item content varies substantially across inventories, the underlying basic interest structure that comprises each RIASEC measure would also vary. When two same-name scales cover different sets of basic interests within the same construct domains, they could still be highly correlated at the between-person level, but the within-person RIASEC score rankings (i.e., interest profiles) could be sensitive to their content variability (Savickas & Taber, 2006). Since interest profiles are commonly used in career counseling settings, content variation could greatly affect practical indications derived from interest results. Despite the foundational importance of item content convergence for research and practice, content validity has received little attention in the field of interest measurement.

Item content is especially relevant in applied settings, such as career guidance, because the interpretation of an individual’s interest results depends on the specific content of an inventory. For example, the Realistic scale of one inventory could contain mainly construction and engineering items, while the Realistic scale from another inventory could include mainly agricultural and outdoor items. An individual who enjoys building or engineering but is not interested in raising dairy cows or working as a park ranger would score high on Realistic using the first inventory, but low on the second. Such differences in content coverage not only influence the client’s Realistic score but could result in a different high-point code leading to different career recommendations.

Basic interests offer a systematic and effective way to examine item content in interest inventories. Basic interests are more homogenous units of interest that can be organized into the structure of Holland’s RIASEC model (Rounds, 1995). Comparing the basic interest coverage of RIASEC scales provides a foundational understanding of what is commonly measured by different inventories, as well as their divergent coverage. If a basic interest is consistently measured by all four inventories, it is central to the construct space of its corresponding RIASEC scale. On the contrary, if RIASEC scales are composed of items from different basic interests, this would indicate a lack of consensus in their content coverage. In this study, we use a recently proposed basic interest framework—the Comprehensive Assessment of Basic Interests (CABIN; Su et al., 2019)—to investigate item coverage. We address the following question:

Research Question 1: To what extent do items in different interest inventories reflect the same basic interest structure within RIASEC scales?

Method

Data Collection

The current study incorporates two types of data: 1) participant survey responses from interest inventories and 2) expert ratings on content coverage of items from corresponding interest inventories. Participant survey responses were adapted from a prior data collection effort for [NAME BLINDED]’s dissertation, which involved the administration of multiple interest inventories to a sample of students that is more representative of typical interest inventory users. Our paper presents analyses based on that sample in addition to ([NAME BLINDED])’s efforts which added expert interest item ratings for the content analysis.

Interest Inventory Participants and Survey Administration

Participants were 327 undergraduates (approximately 80% were freshmen and sophomores) in a large midwestern university. All participants were enrolled in a career development course, an elective course for students who wanted to engage in career exploration and choice processes. Hence, these participants are likely to be representative of students who would typically complete an interest inventory. The course is not required and is open to students from all disciplines.

Participants ranged in age between 17 to 23 years (M = 18.86, SD = 1.18) and were relatively balanced in self-identified sex (43% male, 57% female). A large portion of the students were White/Caucasian (60.3%), followed by 18.4% Black/African American students, 5.3% Asian students, 4.1% multiracial students, .3% Native American students, and 5.6% of students reported “other” for race. In addition, 12.8% of students were Hispanic, and only one participant reported being an international student.

Students were given the set of paper-format surveys during a class meeting with written instructions to take home and complete at their convenience. They were given a date on which their completed packets would be picked up in class but not a particular order they needed to complete the measures. The researchers who took surveys to class for distribution also gave participants oral instructions, and participants were advised not to attempt to complete all the surveys in a single sitting, and to set the task aside if they experienced fatigue or found it difficult to remain focused.

Prior to data analysis, quality control items (embedded within the participants’ response forms and designed to detect random or inattentive responses) were checked. All 327 students in the career development course responded to the survey initially, but seven protocols were identified as suspects of careless responding, and were thus eliminated from the dataset. One participant did not complete the SDS, but other available responses were included in the dataset. This resulted in a final sample size of 319. Although our sample size is relatively small for detecting small correlations among latent factors in our proposed SEM models, post-hoc power analysis suggested that we have close to 1.00 power to detect factor loadings for all trait and method factors, which are the focus of our model result interpretation.

Measures

The present study focuses on four RIASEC measures: Self-Directed Search (SDS; Holland et al., 1994), ACT Revised Unisex Interest Inventory (UNIACT; ACT, 1995), Interest Profiler Short Form (IP Short Form; Rounds et al., 2010) and Strong Interest Inventory (SII; Harmon et al., 1994). These four inventories were chosen because they are among the most used interest inventories, and because they all include a set of scales that specifically measure the RIASEC constructs. Study participants granted permission for their interest inventory results from class to be combined with their responses to the SII, which was administered outside of class. Out of these four measures, we were able to obtain items from the SDS, UNIACT, and IP, but not the SII which is a proprietary measure that does not release its items.

Self-Directed Search (SDS; Holland, 1985)

The SDS was designed by Holland to be a cheap, practical vocational guidance tool that allows users to self-administer, self-score, and self-interpret (Holland et al., 2020). In the SDS, each RIASEC construct is evaluated by 11 activity items in Like/Dislike response format (e.g., repair cars), 11 Competencies items in Yes/No response format (e.g., I can repair furniture), 14 Occupations items in Yes/No response format (e.g., social worker), and 12 Self-Estimates items scoring from 1 (low) to 7 (high) (e.g., scientific ability). An advantage of the dichotomous scoring system is that users can simply add up the “Like” and “Yes” responses for each RIASEC to obtain their interest scores, which requires no transformation or norming on the scores. Hence, participants’ raw scores were also used for their SDS results in this study, and these scores were used to identify their high-point interest code. Scale reliabilities are presented in the inventory manual, and alphas for RIASEC constructs ranged from .83 to .92 (Holland, 1985).

UNIACT-R Level 2 (UNIACT; Swaney et al., 1995)

The UNIACT Level 2 was designed to measure interests for college students and adults, and to only include sex-balanced items. Although the UNIACT effectively reduces sex differences in interest scores, it can reflect a narrower range of RIASEC domains that measure interest in activities with small sex differences (Su et al., 2009). The UNIACT-R contains 90 activity items in total, with 15 items for each RIASEC scale (e.g., operate office machines). All items use the Dislike/Indifferent/Like response format. The UNIACT uses slightly different names for the RIASEC constructs than Holland’s, but the scales are reportedly equivalent: R = Technical, I = Science, A = Arts, S = Social Service, E = Business Contact, C = Business Operations. The UNIACT was scored following instructions in the UNIACT Technical Manual (Swaney et al., 1995). Raw score values range from 15 to 45 and are converted to standardized T-scores (M = 50, SD = 10). Reliability of scales is reported in the manual, and alphas ranged from .77 to .91 (Swaney et al., 1995).

Interest Profiler Short Form (IP Short Form; Rounds et al., 2010)

The IP was developed as one of the Career Exploration Tool sponsored by the U.S. Department of Labor (DOL). One major advantage of the IP is that it is publicly accessible, and the results are directly connected to O*NET’s occupation data for automatically providing users with career recommendations. The IP Short Form contains 60 activity items in total, with 10 items for each RIASEC construct (e.g., manage a retail store). All items use the Dislike/?/Like response format. Raw score values range from 0 to 120. We note that during [NAME BLINDED]’s original data collection, all 180 Interest Profiler items were administered (Lewis & Rivkin, 1999). However, the IP Short Form is the primary form employed in current practice, so we derived the corresponding scores for the Interest Profiler Short-Form using the instructions found on O*NET (Rounds et al., 2021). The reported scale reliability information for the IP Short Form ranged from .78 to .87 (Rounds et al., 2010).

Strong Interest Inventory, (SII; Harmon et al., 1994)

Different from other inventories included in this investigation, the SII was not originally developed to measure general interests like the RIASEC. It started with empirical occupational interest scales which compare the responses of an individual’s interests to those of satisfied employees in specific occupation groups. To make interest results easier to interpret and generalize, basic interest scales were added to the Strong in 1969 (Campbell, 1971). Later, the 1994 edition SII further incorporated Holland’s RIASEC types as an organizing system for its basic and occupational interests. Although the SII has the longest history and has made a profound impact on interest measurement, it can be costly for many job seekers. In addition, the SII has a protected item pool which limits its accessibility to researchers.

The SII evaluates interests using eight types of items and 317 items in total. Five item types are in the Dislike/Indifferent/Like response format: Occupations (135 items), School Subjects (39 items), Activities (46 items), Leisure Activities (29 items), and Types of People (20 items). Two item types ask participants to choose preferences between several options, including Activities (30 items) and Preferences in the Work World (6 items). Personal Characteristics items (12 items) are in the Yes/?/No response format. SII scale score data were taken from reports scored by the publisher, where raw score values were converted to standardized T-scores (M = 50, SD = 10; Harmon et al., 1994). Reliability estimates for RIASEC scales are reported to range from .90 to .94 (Harmon et al., 1994).

Data Analysis

Correlation-Based Convergent Validity

To test hypotheses 1a and 1b, we conducted a nested model comparison among four sets of confirmatory factor analysis (CFA) models on the multitrait-multimethod (MTMM) matrix. Prior to the analyses, we treated random item-level missingness by using full information maximum likelihood to estimate a scale-level correlation matrix with the lavaan package in R. This correlation matrix was then used as data input for further modeling estimation. Model fit for all estimated models was evaluated using comparative fit index (CFI), Tucker–Lewis index (TLI), root-mean-square error of approximation (RMSEA), and standardized root-mean-square residual (SRMR). CFI and TLI values greater than .95, and RMSEA and SRMR values less than .05, suggest good/close model fit; CFI and TLI values greater than .90, RMSEA and SRMR values between .05 to .08 suggest fair/reasonable model fit; and models with RMSEA .08 and .10 indicate mediocre fit (Browne & Cudeck, 1992; Hu & Bentler, 1999).

To compare models, we first fitted a correlated-methods (CM) and a correlated-traits (CT) model. The CM model assumes that variances in observed interest scores are influenced by common latent inventory method factors and unexplained residual variances, whereas the CT model is built on the same assumption regarding latent interest trait factors. Variances across RIASEC scores in the same inventory are assumed to load on one common method factor in the CM model; variances in Realistic scores from all four inventories are assumed to load on one common Realistic trait factor in the CT model. These two models are estimated to compare with fit statistics of models that estimate both trait and method effects. Next, we fit a correlated traits-correlated methods model (CT-CM; Marsh & Grayson, 1995; Widaman, 1985), which assumes that variances in observed interest scores are influenced by both an interest trait and the inventory to which they belong. Traits were allowed to freely correlate, as were methods. No correlations are allowed between trait and method factors to estimate the independent effect of each source. Figure 1 shows a graphical representation of the CT-CM model. Model fit statistics are compared between the CT and CT-CM model to indicate the importance of method effects.

Figure 1.

Correlated Trait – Correlated Method (CTCM) Model of RIASEC Inventory Convergence. Note. SDS = Self-Directed Search; IP = Interest Profiler (Short Form); UNIACT = the Unisex ACT Interest Inventory; SII = Strong Interest Inventory. M1 to M4: different inventories 1 to 4.

Although the CT-CM model separately estimates trait and method effect, the influence of a general trait factor can be confounded by correlated method factors (Eid et al., 2008). Hence, we estimated the third set of CFA models—correlated traits-correlated methods-1 models (CT-C (M-1); Eid et al., 2003)—that offer an alternative view on interest trait and inventory method effects. Similar to the CT-CM model, these models estimate both trait and method effects. Unlike the CT-CM model, the CTC(M-1) model does not suffer from the existence of a potential general factor that can inflate the explained variance of method factors (Eid et al., 2006). The CT-C (M-1) model requires choosing one referent method as the standard for comparison. Because we do not assume one interest inventory to be the gold standard, we estimated four CT-C (M-1) models that each treat one inventory as the referent method. All models were specified and estimated using Mplus version 8.1 (Muthén & Muthén, 1998-2017).

High-Point Code Based Convergence

To test hypotheses 2a and 2b, we examined the level of agreement among high-point codes for each participant on each measure. High-point codes were assigned according to instructions in each inventory’s manual. In general, this involved assigning a letter code based on the person’s highest scores. Tied scores were randomly split to provide the respondent with a single high-point code. In addition to percentages of matched high-point codes between each pair of inventories, we also assessed agreement rates using Cohen’s Kappa (K; Cohen, 1960) and Fliess’s Kappa to better understand the degree of high-point code convergence. These two statistics take into account the chance probability of high-point code agreements (Fleiss, 1981). Kappa values above .60 indicate substantial agreement, values between .41 and .59 indicate fair agreement, and values between .21–.40 indicate poor agreement (Fleiss, 1981).

Item Content-Based Convergent Validity

To address Research Question 1, we first identified content coverage within each RIASEC measure. Traditionally, content validity is established at the beginning of the scale development process by asking subject matter experts to sort items into potential constructs (Hinkin & Tracey, 1999). We adapted this approach to assess convergent validity in content coverage across measures, and four expert raters (two I/O psychology professors and two graduate students) were asked to individually categorize all 366 items from three interest inventories (SDS, UNIACT, IP) into basic interest dimensions. The Strong Interest Inventory items are not open to the public domain and therefore were not included in content analyses.

For the most thorough and updated representation of the interest space, we used the Comprehensive Assessment of Basic Interests (CABIN; Su et al., 2019) which includes 4 to 10 basic interests for each RIASEC domain. The Realistic domain covers 10 basic interests (Agriculture, Animal Service, Athletics, Construction, Engineering, Mechanics/Electronics, Outdoors, Physical/Manual Labor, Protective Service, Transportation/Machine Operation), Investigative covers 4 (Life Science, Mathematics/Statistics, Medical Science, Physical Science), Artistic covers 7 (Applied Arts & Design, Creative Writing, Culinary Art, Media, Music, Performing Arts, Visual Arts), Social covers 8 (Health Care Service, Human Resources, Humanities & Foreign Language, Personal/Service, Religious Activities, Social Science, Social Service, Teaching Education), Enterprising covers 8 (Business Initiatives, Law, Management/Administration, Marketing/Advertising, Politics, Professional Advising, Public Speaking, Sales), and Conventional covers four basic interests (Accounting, Finance, Information Technology, Office Work).

After each rater coded the items independently, acceptable (k > .60) to good (k > .80; Gelfand & Hartmann, 1975; Landis & Koch, 1977) inter-rater agreement was found on all interest types, ranging from k = .75 for Conventional items to k = .96 for Investigative items. Raters met after finishing their coding and discussed items that received different categorizations. The final categorization for each item was determined based on the majority vote. A few items had equal votes for two separate categories, and they were counted as half an item to each category.

Results

Table 2 displays means, standard deviations, and internal consistencies (Cronbach’s alpha and McDonald’s omega hierarchical) of interest scale scores for each inventory. We were not able to calculate internal consistency information for the Strong Interest Inventory because we did not have access to item-level data. The full multitrait-multimethod (MTMM) matrix is also reported in Table 3. Next, we present results for the three sets of analyses.

Table 2.

Descriptive Statistics on Interest Scale Scores.

	SII		UNIACT				IP				SDS
	M	SD	M	SD	Alpha	Omega	M	SD	Alpha	Omega	M	SD	Alpha	Omega
R	42.23	9.37	48.2	9.78	.87	.64	10.74	12.79	.88	.73	17.88	11.37	.95	.81
I	40.93	8.31	48.81	9.43	.92	.73	23	15.19	.86	.68	21.17	9.59	.91	.78
A	44.31	9.09	51.68	9.71	.87	.63	30.38	15.69	.83	.62	24.1	10.65	.91	.60
S	49.48	10.84	50.6	11.13	.83	.56	30.4	15.98	.80	.57	31.39	10.54	.93	.66
E	51.81	10.71	53.28	9.99	.87	.70	25.19	12.91	.78	.52	30.98	10.53	.92	.65
C	47.19	9.7	50.16	9.05	.92	.64	18.05	15.18	.85	.76	21.93	9.73	.93	.67

Note. SDS = Self-Directed Search; IP = Interest Profiler (Short Form); UNIACT = the Unisex ACT Interest Inventory; SII = Strong Interest Inventory. R = Realistic; I = Investigative; A = Artistic; S = Social; E = Enterprising; C = Conventional.

Table 3.

Multitrait-Multimethod (MTMM) Matrix.

	IP_R	IP_I	IP_A	IP_S	IP_E	IP_C	SD_R	SD_I	SD_A	SD_S	SD_E	SD_C	U_R	U_I	U_A	U_S	U_E	U_C	SI_R	SI_I	SI_A	SI_S	SI_E
IP_R	.55
IP_I	.24	.30
IP_A	.03	.26	.30
IP_S	.08	.04	.22	.30
IP_E	.27	.09	.01	.08	.37
IP_C	.78	.48	.21	.00	-.04	.50
SD_R	.36	.74	.32	.09	-.07	.58	.40
SD_I	.53	.52	.76	.91	.38	-.09	.48	.07
SD_A	-.03	.09	.66	.77	.21	.02	-.05	.05	.99
SD_S	-.05	-.09	.05	.69	.64	.25	.04	-.01	.05	.28
SD_E	.02	-.05	-.10	.07	.36	.66	.04	.35	-.06	.59	.48
SD_C	.76	.53	.35	.21	.44	.25	.65	.33	.23	.05	-.03	.05
U_R	.48	.84	.29	.26	-.07	.04	.41	.69	.93	.34	-.49	-.08	.58
U_I	.21	.36	.85	.31	.85	-.04	.80	.72	.76	.98	.03	-.89	.38	.37
U_A	.21	.31	.33	.66	.27	.06	.40	.43	.27	.59	.23	.03	.37	.34	.40
U_S	.00	.03	.23	.33	.69	.37	-.05	.10	.33	.32	.70	.48	.04	.00	.74	.49
U_E	.36	.01	-.06	.04	.48	.75	.05	.05	-.33	.00	.46	.70	.20	-.03	-.67	.07	.55
U_C	.74	.56	.26	.03	.09	.20	.79	.41	.35	-.08	.06	.01	.65	.45	.24	.84	.04	.40
SI_R	.35	.64	.66	.55	.07	.26	.32	.71	.04	.04	.00	.07	.37	.58	.89	.21	.06	.76	.57
SI_I	.06	.21	.66	.31	.23	-.09	-.03	.07	.71	.27	.08	-.10	.24	.27	.71	.36	.21	-.10	.99	.26
SI_A	.09	.69	.23	.72	.24	.09	-.03	.07	.69	.69	.70	.08	.72	.73	.25	.58	.29	.04	.35	.24	.44
SI_S	.05	.02	.53	.20	.67	.30	.00	.06	.07	.68	.58	.34	.54	-.03	.24	.26	.55	.40	.93	.74	.30	.34
SI_E	.26	.08	.06	.06	.46	.66	.06	.68	-.06	.04	.34	.58	.93	.00	-.03	.20	.40	.67	.30	.36	.65	.22	.60

Note. IP = Interest Profiler; SD = Self-Directed Search; U = Revised Unisex ACT Interest Inventory; SI = Strong Interest Inventory.

How Much Do Trait and Method Factors Affect RIASEC Scale Scores?

Table 4 summarizes model fit statistics for the correlated-method (CM) model, correlated-trait (CT) model, the correlated trait-correlated methods (CTCM) model, and the set of correlated trait-correlated methods—1 (CTC(M-1)) Models. Together, these models investigated the degree of independent effects from interest trait and inventory method latent factors on participants’ RIASEC scores. The CM and CT model are the simplest models which assume that variances in interest scores are only decomposed to either method or trait factors and residual error. These two models were compared to test Hypothesis 1a, that interest traits explain more variance in observed RIASEC scores in comparison to inventory methods. The CM model had problems converging due to latent correlations between the different methods exceeding one (RMSEA = .26, CFI = .17, TLI = .07, SRMR = .22). This suggests that there were more latent variables common among the methods (i.e., trait/interests) that were not accounted for by the model. The CT model (RMSEA = .13, CFI = .80, TLI = .77, SRMR = .08) also fitted the data poorly, but the CT model significantly outperformed the CM model, supporting Hypothesis 1a.

Table 4.

Fit Statistics for Competing Confirmatory Factor Analysis Models.

	Chi-square	df	RMSEA	CFI	TLI	SRMR
CM	5413.12	246	.26	.17	.07	.22
CT	1524.83	237	.13	.80	.77	.08
CTCM	683.12	207	.09	.93	.90	.06
CTC(M-1)
SDS-referent	738.29	216	.09	.92	.90	.06
IP-referent	829.76	216	.09	.90	.88	.08
UNI-referent	860.51	216	.10	.90	.87	.09
SII-referent	950.99	216	.10	.89	.86	.15

Note. SDS = Self-Directed Search; IP = Interest Profiler (Short-Form); UNIACT = the Unisex ACT Interest Inventory; SII = Strong Interest Inventory; CT = Correlated Trait model; CTCM = Correlated trait-Correlated method model; CTC(M-1) = Correlated trait-Correlated method minus one model.

To test Hypothesis 1b that method factors would explain a meaningful amount of variance in RIASEC scores, a nested model comparison was conducted. The CTCM model (RMSEA = .09, CFI = .93, TLI = .90, SRMR = .06) performed significantly better than trait-only CT model (Δχ2 (30, N = 219) = 841.71, p < 0.01). This suggests that method factors explain a substantive amount of unique variance in observed RIASEC scores, supporting Hypothesis 1b. In addition to its better model fit, the CTCM model also does not force interpretations to be based on only one interest inventory as the gold standard, but rather how well each inventory assesses each interest type. For these reasons, we retain and discuss factor loadings of the CTCM model.

Table 5 shows the standardized estimates of factor correlations for both latent trait and method factors. Correlations among the RIASEC generally followed the expected pattern according to Holland’s model—adjacent interest categories (e.g., r_RI = .39) were most highly correlated whereas opposite interest categories (e.g., r_RS = −.08) were weakly negatively correlated. Among latent method factors, the SDS showed the weakest correlation with other methods (r’s = .04–.08). The highest method factor correlation was observed among the IP and UNIACT (r = .26). Overall, the correlations among method factors indicate that different inventories affected RIASEC scores in distinct ways (Eid et al., 2008).

Table 5.

Correlations among Latent Trait and Method Factors.

Trait Factors
	R	I	A	S	E
I	.39
A	.10	.12
S	-.08	.07	.17
E	-.07	-.10	.06	.20
C	-.01	-.06	-.17	-.05	.32
Method factors
	SDS	IP	UNI
IP	.05
UNI	.08	.26
SII	.04	.12	.18

Table 6 reports the standardized factor loadings for the traits and method factors, as well as residual variances. In general, the observed interest scores had higher loadings on latent RIASEC \trait factors (range = .57–.93, mean = .77) compared to the latent inventory method factors (range = .25–.74, mean = .42). However, there were some differences in trait and inventory loadings. Across RIASEC traits, interest scores from the SDS showed the strongest average loading on trait factors (.83), as well as the lowest average loading on method factors (.33). These findings coincided with results from the four CTC(M-1) models, among which the model using SDS as the reference method performed the best and met the criteria for acceptable model fit (RMSEA = .09, CFI = .92, TLI = .90, SRMR = .06). In sum, interest traits explained more variance in participants’ scores than inventory methods. Method factors had a stronger influence on the UNIACT and SII, whereas the SDS shared the most variance in interest traits with other interest inventories.

Table 6.

Standardized Parameters for the Correlated Traits-Correlated Methods (CTCM) Model.

	Loadings on Trait Factors
	R	I	A	S	E	C	Row mean
SDS	.93	.80	.84	.85	.82	.72	.83
IP	.77	.84	.84	.84	.77	.74	.80
UNIACT	.57	.79	.87	.62	.81	.87	.75
SII	.81	.65	.71	.73	.61	.63	.69
Column mean	.77	.77	.81	.76	.75	.74	.77
	Loadings on method factors
SDS	.25	.30	.25	.31	.31	.54	.33
IP	.46	.43	.36	.37	.42	.44	.41
UNIACT	.74	.46	.35	.45	.34	.35	.45
SII	.45	.51	.44	.46	.57	.63	.51
Column mean	.48	.43	.35	.40	.41	.49	.42
	Residual variances
SDS	.08	.27	.24	.17	.23	.19	.20
IP	.19	.12	.16	.16	.23	.27	.19
UNIACT	.13	.17	.13	.41	.24	.12	.20
SII	.15	.32	.30	.25	.31	.21	.26
Column mean	.14	.22	.21	.25	.25	.20	.21

Do Respondents Receive the Same High-Point Codes Across Interest Inventories?

Table 7 displays the percentage of high-point code matches and Cohen’s k agreement indices between RIASEC measures. Overall, using scores from two interest inventories, 41–61% (M = 52%) of the participants were assigned the same high-point code. This observed range of high-point code agreement supports the ceiling effect (<70%) proposed in Hypothesis two but rejects the floor effect (>50.8%). This 52% mean hit rate is considerably lower than the hit rate observed between measures using different subsets of the same item pool (69–78%; Rounds et al., 2021). Meanwhile, it is close to the average hit rate of RIASEC inventories in predicting career choices (Hanna & Rounds, 2020).

Table 7.

Percentage of High-point Code Match and Cohen’s k Agreement Indices Among Inventories.

	IP	SDS	UNIACT	SII
IP	--	61%	51%	50%
SDS	.49	--	46%	63%
UNIACT	.39	.35	--	40%
SII	.36	.51	.27	--

Note. Fleiss’ kappa for overall agreement k = .39. Above diagonal are the percentage of participants who have matched high-point codes between pairs of inventories. Below the diagonal are Cohen’s k index of agreement. k values between .41–.59 indicates fair agreement whereas k values between .21–.40 indicates poor agreement.

Correspondingly, there was poor (k = .34) to moderate (k = .45) Kappa agreement rates between pairs of inventories (M = .40). Only two pairs—SDS-SII (63%; k = .51) and SDS-IP (61%; k = .49)—showed fair agreement. The UNIACT-SII pair showed the lowest level of agreement (40%; k =.27). We calculated Fleiss’ kappa (Fleiss, 1981) for multiple-rater (inventory) agreement, which also indicated poor high-point code agreement between the four inventories (k = .39). Thus, the high-point code comparison revealed weaker evidence for convergent validity compared to the correlational analyses. We next conducted item content analyses for the SDS, IP, and UNIACT to investigate whether the RIASEC scales from different inventories contain different structures of basic interests.

Do RIASEC Inventories Share Similar Item Content Coverage?

Table 8 shows the item count for each basic interest dimension in the three inventories, and Figure 2 displays the percentage of items covering each basic interest averaged across the three inventories. Figure 2 clearly shows that certain basic interests occupy large proportions of the construct space within each RIASEC scale—Realistic is mainly characterized by Mechanics/Electronics and Construction/Woodwork; Investigative is mainly characterized by Physical Science; Artistic is mainly characterized by Music, Visual Arts, and Creative Writing; Social is mainly characterized by Teaching/Education, and Social Service; Enterprising is mainly characterized by Management/Administration; Conventional scale is mainly characterized by Accounting and Office Work. Importantly, these common basic interests form a common structure shared by the three RIASEC measures.

Table 8.

Item Count for Basic Interest Categorization and Interrater Agreement.

	IP		SDS		UNIACT
REALISTIC (k = .80)
Mechanics/Electronics	5	50%	13	36%	6	43%
Agriculture	1	10%	1.5	4%	0	0%
Construction/Woodwork	2	20%	15	42%	4	29%
Transportation/Machine operation	1	10%	2.5	7%	1	7%
Outdoors/Nature	1	10%	3	8%	2	14%
Protective service	0	0%	1	3%	0	0%
Others	0	0%	0	0%	2	14%
Investigative (k = .96)
Life science	1	10%	6	17%	6	43%
Physical science	5	50%	11	31%	4	29%
Medical science	3	30%	1	3%	2	14%
Mathematics/Statistics	0	0%	5	14%	0	0%
Others	1	10%	13	36%	3	21%
Artistic (k = .80)
Media	1.5	15%	3	8%	1	7%
Applied arts & design	1	10%	2.5	7%	1.5	11%
Music	3	30%	12	33%	5	36%
Visual arts	2	20%	6	17%	3.5	25%
Performing art	1	10%	4	11%	1	7%
Creative writing	1.5	15%	7	19%	3	21%
Others	0	0%	1.5	4%	0	0%
Social (k = .78)
Teaching/Education	4	40%	8.5	24%	2	14%
Social science	0	0%	6	17%	1	7%
Personal service	0	0%	2	6%	2	14%
Social service	5	50%	14.5	40%	4.5	32%
Human resources	0	0%	1	3%	0	0%
Health care service	1	10%	2	6%	2	14%
Others	0	0%	2	6%	3.5	25%
Enterprising (k = .76)
Management/Administration	4	40%	10	28%	3	21%
Business initiatives	2	20%	2	6%	2	14%
Marketing/Advertising	1	10%	1	3%	1	7%
Professional advising	0	0%	0	0%	2	14%
Public speaking	0	0%	4	11%	0	0%
Sales	1	10%	10	28%	1	7%
Politics	1	10%	2	6%	2	14%
Law	1	10%	0	0%	2	14%
Others	0	0%	7	19%	2	14%
Conventional (k = .75)
Finance	0	0%	5.5	15%	4.5	32%
Accounting	2.5	25%	11.5	32%	6.5	46%
Information technology	2	20%	3	8%	0	0%
Office work	5.5	55%	14	39%	4	29%
Others	0	0%	2	6%	0	0%

Note. SDS = Self-Directed Search; IP = Interest Profiler (Short Form); UNIACT = the Unisex ACT Interest Inventory.

Figure 2.

Averaged Percentage of Items Belonging to Basic Interests. Note. This figure visualizes the average row percentage of basic interest categorization that is reported in Table 8. Sci. = Science; Serv. = Service; Prof. Advise = Professional Advise; Pub. Speak = Public Speaking; IT = Information Technology.

The rest of the construct space consists of peripheral basic interests distributed unevenly across the three inventories. For example, for measuring Investigative, the IP places more emphasis on Medical Science, and the UNIACT places more emphasis on Life Science. The SDS and UNIACT also contain more broad items that could not be categorized into basic interests, especially for Investigative (e.g., read scientific books or magazines), Social (e.g., I find it easy to talk with all kinds of people), and Enterprising scales (e.g., influence others). These items were put into the “other” category for each scale.

Table 9 summarizes key findings of the basic interest coverage across inventories. For this table, we classified basic interests based on their coverage: 1) Core basic interest dimensions are measured by 10% or more of the items in all three inventories; 2) Peripheral basic interests are measured by 10% or more items in two of the three inventories; 3) Under-covered basic interests are measured by less than 10% items in at least two of the three inventories. As noted, the three inventories had reasonable consensus on the most essential (common) basic interests in each RIASEC category, but each had different coverage of peripheral basic interests.

Table 9.

Summary of Content Analysis Findings.

	Core Basic Interests	Peripheral Basic Interest	Under-Covered Basic Interest
Realistic	Mechanics/electronics, construction/Woodwork	Transportation/machine operation, outdoors/nature	Engineering, agriculture, physical/manual labor, athletics, protective service, animal service
Investigative	Physical science	Life science, medical science	Mathematics/Statistics (SDS)
Artistic	Music, visual arts, creative writing	Media, applied arts & design, performing art	Culinary art
Social	Teaching/Education, social service	Health care service	Social science (SDS), personal service (UNIACT), human resources, humanities & foreign language, religious activities
Enterprising	Management/Administration	Business initiatives, sales, politics, law	Professional advising (UNIACT), public speaking (SDS), marketing/Advertising
Conventional	Accounting, office work	Finance	Information technology (IP)

Note. Core Basic Interests are measured by 10% or more of items in all three inventories (SDS, UNIACT, IP); Peripheral Basic Interests are measured by 10% or more of items in two of the three inventories; Under-Covered Basic Interests are measured by 10% or less of items in at least two inventories, and if a basic interest is measured by 10% or more items in one inventory, the specific inventory is noted in parentheses.

In terms of under-covered basic interests, 17 out of the 41 basic interest dimensions were missing or insufficiently covered in at least two of the inventories. For Realistic, there was no coverage of six basic interest scales: Engineering, Agriculture, Physical/Manual Labor, Athletics, Protective Services and Animal Services. For Investigative, only the SDS assessed Mathematics/Statistics. For Artistic, Culinary Art was not measured. For Social, Human Resources, Humanities & Foreign Language, and Religious Activities were missing, while only the SDS contained multiple items measuring Social Science and only the UNIACT covers Personal Service. For Enterprising, only the UNIACT measured Professional Advising, whereas only the SDS measured Public Speaking. For Conventional, Information Technology was missing in all but the IP. Overall, these content gaps highlight new basic interest scales which could be included in future interest inventories. The results also reveal both similarities and differences in the basic interest coverage across RIASEC scales, with important implications for understanding what is measured by existing interest inventories.

Discussion

Vocational interest research and practice rest on a foundational assumption that different inventories assess the same latent constructs and provide similar career exploration information to individuals. The current study critically examined this assumption using four major interest inventories: Self-Directed Search (SDS; Holland et al., 1994), ACT Revised Unisex Interest Inventory (UNIACT; ACT, 1995), Interest Profiler Short Form (IP Short Form; Rounds et al., 2010), and Strong Interest Inventory (SII; Harmon et al., 1994). We used three distinct methods to assess different aspects of construct validity: 1) confirmatory factor analysis (CFA) modeling of multitrait-multimethod (MTMM) matrices, 2) high-point code agreements, and 3) item content coverage. Results from the first two analyses revealed that RIASEC scores from different measures were generally highly correlated, but there was lower than expected high-point code agreement between pairs of inventories. Item content analysis provided one possible explanation for the lack of high-point code convergence by showing that despite interest inventories covering some core basic interests, there are important differences in the content of distinct inventories. We next discuss these findings in greater detail and present theoretical and applied implications.

RIASEC Scores Converged, but High-Point Codes Often Diverged

The first two sets of analyses each painted a distinct picture of convergent validity evidence. First, we estimated correlated traits-correlated measures CFA models, which revealed that interest trait factors had stronger influences on RIASEC scores compared to inventory method factors. This provides evidence for convergence among the SDS, IP, UNIACT, and the SII. Nonetheless, the influence of methods factors on interest scores was also substantial—models with both trait and method factors had a substantially better fit than the trait-only model. The rejection of the trait-only model suggests the final observed score is dependent not only on the trait assessed but also on the inventory used to assess the trait (Eid et al., 2006).

The second method, high-point code analysis, revealed weaker evidence for convergence. High-point codes, reflecting a person’s strongest RIASEC interest, are widely used in vocational interest research and practice (Hanna & Rounds, 2020; Hansen, 2019). Our results indicated that if an individual took two RIASEC inventories, they would be assigned distinct high-point codes about half the time (M_agreement = 52%). This degree of convergence is comparable to the average criterion validity (hit rate) of RIASEC inventories in predicting career choices (Hanna & Rounds, 2020), but is substantially lower than that observed from interest measures using different subsets of the same item pool (69–78%; Rounds et al., 2021).

Item Content Analyses Revealed Convergence, Discrepancies, and Missing Domains

Item content is the basic unit that defines any latent psychological construct. Yet, it has received little attention in the interest assessment literature. Our content analysis distinguished RIASEC scales into three parts: core, peripheral, and missing basic interests. The IP, the SDS, and the UNIACT mostly agreed on the core basic interests within each RIASEC scale. This suggests that interest inventories have a broad consensus on what work activities and occupations are most typical of each RIASEC scale. As noted, Figure 2 provides an overview of common basic interests within each RIASEC scale. This overview serves as a useful tool for researchers to understand what is currently being measured by major RIASEC inventories.

Apart from the core basic interests, the three inventories also included distinctly different basic interest dimensions. We identified these scales as peripheral basic interests. Discrepancies in peripheral basic interests can lead to different overall representations of the RIASEC constructs, potentially explaining the low high-point code convergence among these inventories. For example, respondents who enjoy physical science and mathematics, but are less interested in life or medical science, may receive a higher investigative score using the SDS compared to UNIACT. These differences in scores can lead to variability in ranking RIASEC scores, especially when a person shares similar levels of interest in multiple types. In other words, content differences can add up and ultimately give respondents different interest results.

We also found that a large number of basic interests (17 of 41) are completely missing or insufficiently represented in current RIASEC measures. One possible reason for the incomplete representation of basic interests is that some basic interests reflect more than one RIASEC category. For example, Social Science captures both Investigative and Social types (Su et al., 2019); and Human Resources captures a mix of Social and Enterprising. Including basic interests that tap into more than one RIASEC category can affect the structure of Holland’s model, which is an important quality in RIASEC inventories (Rounds & Day, 1999). However, as a growing number of jobs require an intersection of multiple interest types, ignoring basic interests hinders the effectiveness of interest inventories for providing career guidance.

Implications for Administrating RIASEC Inventories and Measuring Basic Interests

Our content analysis offers applied implications for interpreting results from existing RIASEC inventories and constructing future inventories using basic interests as a structural unit. RIASEC inventories are not interchangeable, and practitioners can benefit by considering the characteristics of target clients for choosing a RIASEC inventory to administer, especially if individuals expressed high interest in areas outside of the core basic interest dimensions (Lowman, 2022). The divergent results from our two quantitative analyses suggest that interest inventory users should be cautious when interpreting high-point codes. Although many practitioners or inventory instructions indeed walk clients through their entire interest profiles, it is also important to communicate the complexity of individual interests and the fact that people rarely fit neatly into one interest type (Lowman, 2022). Hence, interest inventory users should be reminded that interest assessments are not diagnostic, and they need to remain flexible in thinking about their interests. Flexible thinking is particularly important when users take multiple self-interpreted interest measures—they need to be made aware that if they receive different interest codes from two inventories over time, it does not necessarily reflect true changes in their interests. To help individuals correctly interpret and further understand their interest results, existing RIASEC assessments should include relevant information about what basic interests are being measured and those not being measured under each construct.

It is also important for researchers to consider differences between RIASEC inventories when including interest scales in studies. Differences in item content from inventories can yield different scale scores and consequently different study results. For example, previous research suggests variability between interest inventories when examining their fit to Holland’s circumplex, which results in different degrees of structural validity (Armstrong et al., 2003; Rounds & Tracey, 1996; Tracey & Rounds, 1993). Previous meta-analyses on the criterion-related validity of vocational interests also suggest differences in validities obtained from different measures. For example, the relations between interest fit with task performance (ρ = .20 to .31; Nye et al., 2017) and interest fit and job satisfaction (ρ = .11 to .24; Hoff et al., 2020) vary depending on the inventory. Some variation of the observed differences in criterion validity of interest inventories can be explained by each measure’s unique history. For example, the SII was developed to emphasize predictive validity using empirically derived occupational scales. However, variation in item content coverage can impact all aspects of validity for an inventory. Hence, researchers should be aware of what basic interests are included by different inventories when selecting the most relevant measure. For example, using only the IP to assess the interests of financial analysts would fail to assess the basic interests most likely to be relevant to the sample (i.e., finance and statistics/mathematics). To address this gap, the researcher could supplement the interest measure with finance and statistics basic interest scales from a public domain basic interest inventory (e.g., Liao et al., 2008).

For both practitioners and researchers who plan to assess basic interests, we offer the following recommendations for selecting basic interest scales according to the targeted respondents. To provide career or educational guidance for students, we suggest including core basic interests with the addition of basic interests that reflect important instructional programs. For example, including all basic interests under the Investigative type is desirable because each can be directly linked to college majors. To offer career guidance for the general public, we suggest including basic interests that map onto fast-growing occupations in the workforce. For instance, according to the U.S. Bureau of Labor Statistics (BLS), retail trade and healthcare each makeup roughly 10% and 12% of the service-providing industry (U.S. Bureau of Labor Statistics, 2021). Therefore, it is important to include sales and healthcare service dimensions in Enterprising and Social scales, respectively. Another example is Information Technology, which connects to conventional interests and is projected to grow 12% by 2029 (U.S. Bureau of Labor Statistics, 2022).

Altogether, our results emphasize the importance of assessing convergent validity evidence in multiple, distinct ways. CFA modeling on the MTMM matrix effectively compares the independent effects of traits and methods on scores, and they help assess whether different inventories are measuring the same latent constructs (Eid et al., 2008). Yet, they provide little information about whether different inventories yield the same results in practice. Inventories may be fairly robust in measuring the same underlying constructs, but subtle variations in item content can yield divergent results that lead to different high-point codes. Construct validity research should therefore take into consideration how scores are used and interpreted in practice. Otherwise, the real-world impact of using different measures to assess the same construct could be masked by high correlation-suggested convergence.

Strengths, Limitations, and Future Directions

To our knowledge, the current study is the most comprehensive assessment of convergence and content validity among vocational interest inventories. We quantified the degree of convergence among major interest inventories using correlation-based and high-point code-based approaches, and we also examined item content similarities and differences across RIASEC scales. Each method revealed distinct aspects of convergent validity that provide important implications for understanding what is being measured by RIASEC inventories. However, there are several noteworthy limitations of the current research.

First, for our quantitative analyses, we adapted prior data on four major interest inventories. Although our adaptation allows us to include the most widely used public domain inventory today—the Interest Profiler Short Form, there are other interest inventories that we did not analyze (Hansen, 2019). In addition, we were unable to obtain item-level data from the Strong Interest Inventory and calculate its scale reliabilities in our study. Future researchers who have access to all items from multiple interest measures could replicate our quantitative analyses on the item-level data. Furthermore, future content validity studies on interest measurement could benefit from analyzing a wider selection of publicly accessible interest items, particularly in examining the consistency of core basic interest dimensions identified in the current study.

Second, in addition to differential coverage, convergence also depends on other factors relevant to the development of each inventory. For example, interest inventories can differ in their item selection to reduce sex differences, the use of normed versus raw score reporting, items written for different levels of education, and item type (e.g., occupations, activities, and school subjects). Notably, the UNIACT selected items with small sex differences, which potentially explains its stronger method effect on Realistic and Social, the two scales with the largest sex differences (Su et al., 2009). Future studies could further assess the effects of specific elements on convergence among interest scores.

Third, there is currently no uniform consensus on the number and structure of basic interests. Hence, our content analysis results, which are based on the Comprehensive Assessment of Basic Interest (CABIN; Su et al., 2019), should be interpreted as constructive, not definitive. With these limitations in mind, we put forth our key message for improving interest assessment: As the world of work changes, vocational interest inventories must update their representation of interest dimensions through bottom-up processes, capturing the types of jobs most relevant in the present and future labor market. In addition, all methods of construct validity, including content and convergent, should be regularly assessed to keep interest inventories up to date. Thus, this study provides a benchmark for future revisions on the construct validity of interest inventories.

Conclusion

The current study assessed construct validity among major interest inventories using three methods, each providing unique contributions to better understand interest measurement. The results showed that major interest inventories reflect similar RIASEC traits in general, but they less often provide consistent high-point codes to respondents. Moreover, our analysis explored why interest inventories are not interchangeable by showing their distinct coverage of peripheral basic interests and missing basic interests. Overall, our findings help integrate the theoretical understanding of Holland’s (1997) RIASEC constructs with important implications for improving existing RIASEC inventories, as well as developing the next generation of basic interest-based assessments.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author Note

Chu Chu and Marry T. Russell share joint first authorship of this manuscript. Our study utilizes Mary Russell’s dissertation (Russell, 2008), Assessing Vocational Interests: Convergence and Divergence of Inventories and Informants, as a starting point. Chu Chu conducted reanalyses of Russell’s dataset, led efforts to expand the scope of the paper to include item content analysis of basic interests, and prepared this manuscript for publication.

References

American College Testing Program (1995). Technical manual: Revised unisex edition of the ACT interest inventory (UNIACT). American College Testing Program.

Armstrong

P. I.

Allison

Rounds

(2008). Development and initial validation of brief public domain RIASEC marker scales. Journal of Vocational Behavior, 73(2), 287–299. https://doi.org/10.1016/j.jvb.2008.06.003.

Armstrong

P. I.

Hubert

Rounds

(2003). Circular unidimensional scaling: A new look at group differences in interest structure. Journal of Counseling Psychology, 50(3), 297–308. http://dx.doi.org/10.1037/0022-0167.50.3.297.

Blakeney

R. N.

Matteson

M. T.

Holland

T. A.

(1972). A research note on the new SVIB Holland scales. Journal of Vocational Behavior, 2(3), 239–243. https://doi.org/10.1016/0001-8791(72)90029-2.

Browne

M. W.

Cudeck

(1992). Alternative ways of assessing model fit. Sociological methods & Research, 21(2), 230–258. https://doi.org/10.1177/0049124192021002005.

Campbell

D. P.

(1971). Handbook for the strong vocational interest blank. CA: Stanford University Press.

Campbell

D. P.

Borgen

F. H.

(1999). Holland's theory and the development of interest inventories. Journal of Vocational Behavior, 55(1), 86–101. http://dx.doi.org/10.1006/jvbe.1999.1699.

Campbell

D. T.

Fiske

D. W.

(1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. http://dx.doi.org/10.1037/h0046016.

Campbell

D. P.

Holland

(1972). A merger in vocational interest research: Applying holland's theory to strong's data. Journal of Vocational Behavior, 2(4), 353–376. http://dx.doi.org/10.1016/0001-8791(72)90012-7.

10.

Cohen

(1960). A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20, 37–46. http://dx.doi.org/10.1177/001316446002000104.

11.

Dixon

Johnston

(2019). Content validity of measures of theoretical constructs in health psychology: Discriminant content validity is needed. British Journal of Health Psychology, 24(3), 477–484. https://doi.org/10.1111/bjhp.12373.

12.

Eid

Lischetzke

Nussbeck

F. W.

(2006). Structural equation models for multitrait-multimethod data. In Eid

Diener

(Eds), (pp. 283–299). Handbook of multimethod measurement in psychology; Handbook of multimethod measurement in psychology. American Psychological Association. Chapter xiv, 553 Pages) http://dx.doi.org/10.1037/11383-020.

13.

Eid

Lischetzke

Nussbeck

F. W.

Trierweiler

L. I.

(2003). Separating trait effects from trait-specific method effects in multitrait-multimethod models: A multiple-indicator CT-C(M-1) model. Psychological Methods, 8(1), 38–60. http://dx.doi.org/10.1037/1082-989X.8.1.38.

14.

Eid

Nussbeck

F. W.

Geiser

Cole

D. A.

Gollwitzer

Lischetzke

(2008). Structural equation modeling of multitrait-multimethod data: Different models for different types of methods. Psychological Methods, 13(3), 230–253. http://dx.doi.org/10.1037/a0013219.

15.

Fabry

Blake

Seran

(1978). A construct validation study of the ACT Interest Inventory with high school students. Measurement and Evaluation in. Guidance, 10(4), 233-236. https://doi.org/10.1080/00256307.1978.12022136.

16.

Fleiss

J. L.

(1981). Statistical methods for rates and proportions. John Wiley & Sons.

17.

Furr

R. M.

Bacharach

V. R.

(2006). Validity: Estimating and evaluating convergent and discriminant validity evidence. Psychometrics: An Introduction, 191–235.

18.

Gelfand

D. M.

Hartmann

D.P.

(1975). Child behavior analysis and therapy. New York: Pergamon.

19.

Hanna

Rounds

(2020). How accurate are interest inventories? A quantitative review of career choice hit rates. Psychological Bulletin, 146(9), 765–796. http://dx.doi.org/10.1037/bul0000269.

20.

Hansen

J. C.

(2019). Interest Inventories. In Goldstein

Allen

D. N.

DeLuca

(Eds), (4th ed., pp. 169–190). 4th ed.; Handbook of psychological assessment. Elsevier Academic Press. http://dx.doi.org/10.1016/B978-0-12-802203-0.00006-7.

21.

Hanson

G. R.

(1974). Assessing the career interests of college youth: Summary of research and applications. ACT Research Reports, 67, 76.

22.

Hanson

G. R.

Lamb

R. R.

English

(1974). An analysis of Holland’s interest types for women: A comparison of the Strong-Holland and the ACT Vocational Interest Profile scales for women. Journal of Vocational Behavior, 4(2), 259–269. https://doi.org/10.1016/0001-8791(74)90109-2.

23.

Harmon

L. W.

DeWitt

D. W.

Campbell

D. P.

Hansen

J. I. C.

(1994). Strong interest inventory: Applications and technical guide: Form T317 of the strong vocational interest blanks. Stanford University Press.

24.

Harmon

L. W.

Zytowski

D. G.

(1980). Reliability of holland codes across interest measures for adult females. Journal of Counseling Psychology, 27(5), 478–483. http://dx.doi.org/10.1037/0022-0167.27.5.478.

25.

Hinkin

T. R.

Tracey

J. B.

(1999). An Analysis of Variance Approach to Content Validation. Organizational Research Methods, 2(2), 175–186. https://doi.org/10.1177/109442819922004.

26.

Hoff

K.A.

Chu

Einarsdóttir

Briley

D.A.

Hanna

Rounds

(2022). Adolescent vocational interests predict early career success: Two 12-year longitudinal studies. Applied Psychology, 71, 49-75. https://doi.org/10.1111/apps.12311.

27.

Hoff

K. A.

Song

Q. C.

Wee

C. J.

Phan

W. M. J.

Rounds

(2020). Interest fit and job satisfaction: A systematic review and meta-analysis. Journal of Vocational Behavior, 103503. https://doi.org/10.1016/j.jvb.2020.103503.

28.

Holland

J. L

(1973). Making vocational choices. Prentice-Hall.

29.

Holland

J. L

(1985). Making vocational choices: A theory of vocational personalities and work environments. Prentice-Hall.

30.

Holland

J. L

(1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Psychological Assessment Resources.

31.

Holland

J. L.

Fritzsche

Powell

(1994). Self-directed Search: Technical manual. Psychological Assessment Resources.

32.

Holland

J. L.

Rayman

J. R.

Gottfredson

G. D.

(2020). My life with a theory: John L. Holland's autobiography and theory of careers. National Career Development Association.

33.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling, 6(1), 1–55. http://dx.doi.org/10.1080/10705519909540118.

34.

Lamb

R. R.

Prediger

D. J.

(1981). Technical report for the Unisex Edition of the ACT Interest Inventory (UNIACT). American College Testing Program.

35.

Landis

Koch

(1977). The measurement of observer agreement for categorical data. Biometrics, 33(1), 159–174. https://doi.org/10.2307/2529310.

36.

Lewis

Rivkin

(1999). Development of the O* NET interest profiler. National Center for O* NET Development.

37.

Liao

Armstrong

P. I.

Rounds

(2008). Development and initial validation of public domain basic interest markers. Journal of Vocational Behavior, 73(1), 159–183. http://dx.doi.org/10.1016/j.jvb.2007.12.002.

38.

Lowman

R. L.

(2022). Career assessment: Integrating interests, abilities, and personality. American Psychological Association. http://dx.doi.org/10.1037/0000254-000.

39.

Lowman

R. L.

Palmer

L. K.

Santana

Abott

(2003). Executive and career assessment: Can alternative occupational interest measures Be used interchangeably? The Psychologist-Manager Journal, 6(2), 65–78. http://dx.doi.org/10.1037/h0095927.

40.

Marsh

H. W.

Grayson

(1995). Latent variable models of multitrait-multimethod data. In Hoyle

R. H.

(Ed), (pp. 177–198). Structural equation modeling: Concepts, issues, and applications; structural equation modeling: Concepts, issues, and applications. Sage Publications, Inc.

41.

Mount

K. M.

Barrick

M. R.

Scullen

S. M.

Rounds

J. B.

(2005). Higher order dimensions of the big five personality traits and the big six interest types. Personnel Psychology, 58, 447–478. http://dx.doi.org/10.1111/j.1744-6570.2005.00468.x.

42.

Muthén

L.K.

Muthén

B.O.

(1998–2017). Mplus user’s guide (Eighth Edition). Muthén & Muthén.

43.

Nye

C. D.

Rounds

Drasgow

(2012). Vocational interests and performance: A quantitative summary of over 60 years of research. Perspectives on Psychological Science, 7(4), 384–403. http://dx.doi.org/10.1177/1745691612449021.

44.

Nye

C. D.

Rounds

Drasgow

(2017). Interest congruence and performance: Revisiting recent meta-analytic findings. Journal of Vocational Behavior, 98, 138–151. http://dx.doi.org/10.1016/j.jvb.2016.11.002.

45.

Passler

Beinicke

Hell

(2015). Interests and intelligence: A meta-analysis. Intelligence, 50, 30–51. http://doi.org/10.1016/j.intell.2015.02.001.

46.

Prediger

D. J.

Swaney

K. B.

(2004). Work task dimensions underlying the world of work: Research results for diverse occupational databases. Journal of Career Assessment, 12(4), 440–459.

47.

Rounds

(1995). Vocational interests: Evaluating structural hypotheses. In Lubinski

D. J.

Dawis

R. V.

(Eds), (pp. 177–232). Assessing individual differences in human behavior: New concepts, methods, and findings; assessing individual differences in human behavior: New concepts, methods, and findings. Davies-Black Publishing.

48.

Rounds

Day

S. X.

(1999). Describing, evaluating, and creating vocational interest structures. In Savickas

M. L.

Spokane

A. R.

(Eds), (pp. 103–133). Vocational interests: Meaning, measurement, and counseling use; vocational interests: Meaning, measurement, and counseling use. Davies-Black Publishing.

49.

Rounds

Hoff

K. A

Lewis

(2021). O*NET® interest profiler manual. U.S. Department of Labor National O*NET Resource Center. https://www.onetcenter.org/reports/IP_Manual.html.

50.

Rounds

(2014). The nature and power of interests. Current Directions in Psychological Science, 23(2), 98–103. http://dx.doi.org/10.1177/0963721414522812.

51.

Rounds

Lewis

Rivkin

(2010). O* NET interest profiler short form psychometric characteristics: Summary. National Center for O* NET Development.

52.

Rounds

Lewis

Rivkin

(2010). O* NET interest profiler short form psychometric characteristics: Summary. Raleigh, NC: National Center for O* NET Development.

53.

Rounds

Tracey

T. J.

(1996). Cross-cultural structural equivalence of RIASEC models and measures. Journal of Counseling Psychology, 43(3), 310–329. http://dx.doi.org/10.1037/0022-0167.43.3.310.

54.

Rounds

Walker

C. M.

Day

S. X.

Hubert

Lewis

Rivkin

(1999). O* NET Interest Profiler: Reliability, validity, and self-scoring . Raleigh, NC: National Center for O* NET Development.

55.

Assessing vocational interests: Convergence and divergence of inventories and informants (Order No. AAI3301220) Russell

(2008). Available from APA PsycInfo®. (621739401; 2008-99150-261). https://search.proquest.com/dissertations-theses/assessing-vocational-interests-convergence/docview/621739401/se-2?accountid=14553.

56.

Savickas

M. L.

Taber

B. J.

(2006). Individual differences in RIASEC profile similarity across five interest inventories. Measurement and Evaluation in Counseling and Development, 38(4), 203–210. https://search.proquest.com/scholarly-journals/individual-differences-riasec-profile-similarity/docview/621116351/se-2?accountid=14553.

57.

Savickas

M. L.

Taber

B. J.

Spokane

A. R.

(2002). Convergent and discriminant validity of five interest inventories. Journal of Vocational Behavior, 61(1), 139–184. http://dx.doi.org/10.1006/jvbe.2002.1878.

58.

Stoll

Einarsdóttir

Song

Q. C.

Ondish

Sun

J. J.

Rounds

(2020). The roles of personality traits and vocational interests in explaining what people want out of life. Journal of Research in Personality, 86, 103939. https://doi.org/10.1016/j.jrp.2020.103939.

59.

Nye

C. D.

(2017). Interests and person-environment fit: A new perspective on workforce readiness and success. In Burrus

Mattern

K. D.

Naemi

B. D.

Roberts

R. D.

(Eds), Building better students: Preparation for the workforce; Building better students: Preparation for the workforce (pp. 177–206). Oxford University Press.

60.

Rounds

Armstrong

P. I.

(2009). Men and things, women and people: a meta-analysis of sex differences in interests. Psychological Bulletin, 135(6), 859–884. http://dx.doi.org/10.1037/a0017364.

61.

Tay

Liao

Zhang

Rounds

(2019). Toward a dimensional model of vocational interests. Journal of Applied Psychology, 104(5), 690–714. http://dx.doi.org/10.1037/apl0000373.

62.

Swaney

K. B.

Lamb

Prediger

American College Testing Program (1995). Technical manual: Revised unisex edition of the ACT interest inventory (UNIACT). ACT.

63.

Tracey

T. J.

Rounds

J. B.

(1993). Evaluating holland's and gati's vocational-interest models: a structural meta-analysis. Psychological Bulletin, 113(2), 229–246. http://dx.doi.org/10.1037/0033-2909.113.2.229.

64.

U.S. Bureau of Labor Statistics . (2021, September 8). Employment by Major Occupational Group. U.S. Bureau of Labor Statistics. Retrieved March 29, 2022, from https://www.bls.gov/emp/tables/emp-by-major-occupational-group.htm

65.

U.S. Bureau of Labor Statistics . (2022, March 4). Employment and earnings table B-1A. U.S. Bureau of Labor Statistics. Retrieved March 29, 2022, from https://www.bls.gov/web/empsit/ceseeb1a.htm

66.

Van Iddekinge

C. H.

Putka

D. J.

Campbell

J. P.

(2011). Reconsidering vocational interests for personnel selection: the validity of an interest-based selection test in relation to job knowledge, job performance, and continuance intentions. Journal of Applied Psychology, 96(1), 13–33. http://dx.doi.org/10.1037/a0021193.

67.

Widaman

K. F.

(1985). Hierarchically nested covariance structure models for multitrait-multimethod data. Applied Psychological Measurement, 9(1), 1–26. http://dx.doi.org/10.1177/014662168500900101.