Abstract
Since their introduction, interests have been important to the field of vocational psychology. Within career counseling, they often serve as a starting point to career exploration. The changing nature of the world of work means that accurate assessment is even more imperative. There remain a number of inventories to help assess this construct. The Occupational Information Network–Interest Profiler–Short Form (O*NET-IP-SF) is a public domain assessment utilized to measure Holland’s six occupational profiles: realistic, investigative, artistic, social, enterprising, and conventional. Using confirmatory factor analysis, this study examined the structural integrity of the O*NET-IP-SF and interpretive potential using a regionally and racially diverse sample of American college students. Results suggest that O*NET-IP-SF possesses poor fit with a six-factor structure and that modifications are needed to reach acceptable levels of scale performance. Implications for researchers and career counselors are discussed.
Keywords
Matching information about one’s self to the world of work remains an approach that has dominated vocational psychology since the turn of the 20th century (e.g., Parsons, 1909). Interests provide a means to understand worker attitudes and aspirations (Larson, Rottinghaus, & Borgen, 2002) as well as information about working environments (Gottfredson & Holland, 1996). Work has shifted toward a more dynamic and adaptive nature (Savickas, 2000), and career development belongs more to individuals than to corporations (Duarte, 2004). Even within a dynamic world of work, vocational interests remain integral for workers attempting to identify, obtain, maintain, and disengage from work. Individuals consult these vocational interests to help make better career decisions more appropriately fitted to their needs. In light of this, it is crucial that accurate assessment of these interests occurs.
Krieshok, Black, and McKay (2009) asserted that vocational psychology is transitioning from ‘“a place where it’s all about the match’, to one that is ‘all about adapting to change’” (p. 275). However, in this world of work, matching assessments are now dual-purposed. They are utilized not only to match individuals to careers because of shared characteristics but also to stimulate learning in the individual (Krumboltz, 2009). The adaptable worker reads and reacts to unpredictable changes and adjustments that occur within the work role (Savickas, 1997). Assessment also facilitates exploration, which may play a role in coping with unpredictability (Blustein, 1997). It may also promote learning about areas of work that may offer a better fit to the needs, values, and vocational interests of the worker. Interests remain important to the present and the future of vocational psychology. Given their importance, there is a need (Juntunen & Bailey, 2013) to ensure the accuracy of public domain interest instruments. This article will provide a brief review of interest instruments before focusing on the Occupational Information Network Interest Profiler–Short Form (O*NET-IP-SF; Rounds, Su, Lewis, & Rivkin, 2010).
Assessment of Interests
Rounds and Tracey (1996) summarized 10 different types of interest assessments and additional cross-cultural adaptations’ fit to different models (e.g., Gati’s partition model). However, most interest assessments focus on Holland’s typography using the six general occupational realistic, investigative, artistic, social, enterprising, and conventional (RIASEC) themes (e.g., Donnay, Morris, Schaubhut, & Thompson, 2005). Hansen (2013) listed the most popular assessments of interests: the Self-directed Search (Holland, 1994), the Strong Interest Inventory (Donnay et al., 2005), the Campbell Interest and Skill Survey (Campbell, Hyne, & Nilnsen, 1992), and the Occupational Information Network–Interest Profiler (O*NET-IP; Lewis & Rivkin, 1999).
Three of these four assessments are proprietary—meaning the instruments carry copyrights and a cost per use. Issues exist with proprietary instruments including the reduced opportunity to improve scale quality (Goldberg, 1999). Additionally, despite the 4–6 million paid inventories sold every year, there remains a dearth of information regarding domestic ethnic samples (Rounds & Tracey, 1996). Lastly, proprietary instruments contrast with Miller’s (1969) charge to give psychology “away to those who really need it—and that includes everyone” (p. 1071). Within vocational psychology, this call to attend to others, especially those of limited financial means, has been issued (e.g., Blustein, 2006).
The fourth of the most popular interest assessments, the O*NET-IP (Lewis & Rivkin, 1999), utilizes a public domain approach in contrast to the proprietary interest inventories. This instrument is featured on the O*NET Resource Center website (O*NET Resource Center—Interest Profiler) and the Interest Item Pool (IIP) website (Liao, Armstrong, Rounds, & Su, 2007). The IIP is composed of 338 items across 31 basic interest domains (Armstrong, Allison, & Rounds, 2008); additionally, it includes the O*NET-IP-SF (Rounds et al., 2010). The IIP offers two assessment forms for activities and occupations (Liao, Armstrong, & Rounds, 2008). Activities focus on tasks one may complete (e.g., “lay brick or tile” and “sell merchandise at a department store”). Occupations focus on more popular labels used to describe positions (e.g., “occupational therapist” and “payroll and timekeeping clerk”).
Due to its brevity and availability, the O*NET-IP-SF offers substantial opportunity for broad use across a variety of vocational assessment domains. During its validation, Rounds, Su, Lewis, and Rivkin (2010) provided initial validity and reliability evidence for the O*NET-IP-SF in two samples (n = 1,061 for the development sample and n = 132 for the validation sample). During the O*NET-IP-SF’s development, domain coefficient αs were .78 (realistic, artistic, and social), .82 (investigative), .87 (enterprising), and .83 (conventional) with a mean of .81 in the developmental sample. In the stability sample, domain coefficient αs were .82 (realistic), .86 (investigative), .88 (artistic), .85 (social), .83 (enterprising), and .90 (conventional). The mean coefficient α across scales was .86. In the validation sample, test–retest correlations were .79 (realistic), .78 (investigative), .82 (artistic), .85 (social), .82 (enterprising), and .86 (conventional); there was a mean test–retest of .82. Scale correlations with the Interest Profiler–Long Form ranged from .90 (social) to .95 (conventional) for the correct match (e.g., Realistic Scale to Realistic Scale), while correlations for scales that were not similar (e.g., Realistic Scale to Conventional Scale) ranged from .12 (investigative to conventional) to .48 (investigative to artistic). The O*NET-IP-SF had good convergent and divergent validity evidence as well with correlations for correctly matched domains ranging from .74 (social) to .82 (conventional) and dissimilar matches ranged from .12 (realistic to social) to .48 (social to enterprising).
Current Issue and Aims
Unfortunately, no evaluation exists regarding the factor structure of the O*NET-IP-SF and coefficient α provides only a crude estimation of internal reliability (Brown, 2015). This lack of a stronger psychometric evaluation remains troubling, particularly since the items selected for the O*NET-IP-SF have recently been used as the exclusive content for another briefer, vocational instrument (e.g., Rounds, Ming, Cao, Song, & Lewis, 2016). Moreover, initial career counseling often remains limited to the assessment of interests, which remains a process which may be dictated by time and financial limitations (Hansen, 2013). As such, a need exists to ensure that instruments chosen to measure Holland’s six vocational personality profiles reflect the strongest psychometric form possible. To address these concerns, this study evaluated the structural form of the O*NET-IP-SF using confirmatory factor analysis with the intent of strengthening the available validity evidence for a popular instrument derived from the IIP.
Method
Participants
A total of 397 college students (95 male; 23.9%) participated in this study. Participants were recruited from two separate universities located in the United States: a large midwestern university (n = 256; 63 males, 24.8%) and a midsized southern university (n = 141; 32 males, 23.7%). Fifty-eight students in the midwestern sample and 69 students in the southern university sample reported being first-generation college students (22.7% and 44.2%). The sample included both traditional and nontraditional college students given the age range (18–30+ in both samples, with 34.5% of the sample reporting being 25 years or age or older) and years in college reported (1–6+ years, with 30% of the sample reporting being in at least their sixth year of college). Means and standard deviations cannot be calculated as data were collected in a range format. These numbers do not match the sample perfectly, as all participants were given the option to decline reporting demographics. Participants completed the O*NET-IP-SF and demographic information via an online survey system in exchange for extra credit or as part of course credit.
Instrumentation
O*NET-IP-SF
The O*NET-IP-SF (Rounds et al., 2010), a popular public domain vocational profiling instrument prepared for the U.S. Department of Labor, is featured on the O*NET Resource Center website (n.d.). Using a 5-point Likert-type scale (strongly like, like, unsure, dislike, or strongly dislike) individuals indicate their interest in specific work activities over 60 total items. Relationships between the scales of this instrument and those of its parent instrument (e.g., Interest-Finder RIASEC; Wall & Baker, 1997) demonstrated strong convergent validity. Reliability estimates showed high coefficient α and test–retest reliability values exceeding .80 on average in both. Descriptive statistics for the scales, as well as coefficient αs of this sample, are provided in Table 1.
Descriptive Statistics of the Occupational Information Network–Interest Profiler–Short Form Domains.
Note. SD = standard deviation; M = mean.
Methods and Planned Analysis
Confirmatory factor analyses (CFA), as well as the calculation of descriptive statistics, were planned for the O*NET-IP-SF instrument in a model containing 10 items on each of the six factors. If difficulties arose in the combined model, independent analyses on each RIASEC domain were planned. Analyses were completed using Mplus 7 (Muthén & Muthén, 2012). To assess appropriateness of the tested models, interpretation of CFA results planned to use Hu and Bentler’s (1999) guidelines for model fit. These guidelines suggest excellent fit is indicated by a root mean square error of approximation (RMSEA) of about .06 or below as well as a comparative fit index (CFI) and a Tucker–Lewis index (TLI) of .95 or greater.
Results
The initial CFA of the O*NET-IP-SF utilized all 60 items loading onto the six RIASEC domains and this model fit poorly, χ2(1,695) = 3,839.78; CFI = .70, TLI = .69, RMSEA = .08 (.08–.09), standardized root mean square residual = .10. Given this poor fit, separate confirmatory analyses were conducted on each RIASEC domain (Table 2), as each domain is intended to measure a discrete latent construct. Beginning with the base model for each RIASEC domain, modification indices and item loadings were evaluated to determine indicators of misfit that could be corrected if the initial domain model did not provide evidence of good fit. Individual item loadings were not problematic within any of the models. However, domains frequently needed residual corrections added to improve fit due to correlation between item level responses not being explained by the latent vocational interest domain. Corrections were conducted one at a time until acceptable fit for an individual domain was reached. Modification indices were selected based on their corrective influence on the χ2 value. In some instances, an item needed numerous correlated residuals with other items within the scale. Completing this would greatly complicate the interpretability of the model and, rather than adding multiple residual correlations, problematic items were removed. Excluded items generally were suggested as being significantly related to at least 40% of other items on the scale.
Fit Indices for Domains of the Occupational Information Network–Interest Profiler–Short Form Across Corrective Stages.
Note. CR = correlated residual correction; IR = item removal; TLI = Tucker–Lewis index; RMSEA = root mean square error of approximation; SRMR = standardized root mean square residual; CFI = comparative fit index.
While adding correlated residuals is not preferred (Little, 2013), their addition remains useful in model improvement efforts if those residual correlations are grounded in theory or make conceptual sense. In addition to steps ensuring empirical significance for corrections, steps were taken to ensure that the included residual relationships made conceptual sense. In order to help ensure that corrections were appropriate and not merely sample specific, item-level wording comparisons were also conducted. Each time that modification indices were incorporated, thematic components not associated with the broader interest domain remained present. For instance, realistic domain Items 1 (build kitchen cabinets) and 2 (lay brick or tile) were thematically associated with construction work, while Items 14 (raise fish in a fish hatchery) and 50 (put out forest fires) deal with forestry. Our approach to correction ensures that items share both a general domain-inspired relationship and a more specific context. Fit statistics for domains are reported in Table 2, both with initial domain models and with each correction step taken following either the inclusion of a modification index or the exclusion of a poorly loading item. Standardized item loadings are presented in Figure 1 for the CFA results of each RIASEC domain model. Correlations between the RIASEC domains in both the base O*NET-IP-SF and the final CFA model are presented in Figure 2.

Confirmatory factor analysis results of final realistic, investigative, artistic, social, enterprising, and conventional (RIASEC) domains. RIASEC domains are abbreviated in the model above by a single letter: R = realistic, I = investigative, A = artistic, S = social, E = enterprising, and C = conventional.

Correlations between Occupational Information Network–Interest Profiler–Short Form (O*NET-IP-SF) domains. Panel (a) contains correlations from the realistic, investigative, artistic, social, enterprising, and conventional domains as recommended for standard scoring of the O*NET-IP-SF, while panel (b) shows final model correlations following confirmatory factor analyses modifications conducted within this study. All correlations are significant at p < .05.
Following evaluation of each individual domain, the final six factor model of the O*NET-IP-SF was evaluated again using these corrected domains, χ2(300) = 2,899.25, CFI = .88, TLI = .87, RMSEA = .07 (.07–.08). Although this model did not reach these studies a priori definition of excellent fit (e.g., Brown, 2015), it did show substantial improvement over the initial model due to several domains achieving acceptable fit when evaluated independently. Despite lower than desired CFI and TLI statistics, this final model produced an RMSEA value unlikely to indicate problems in model misspecification (Gonzales & Ferrer, 2014).
Discussion
This study was the first to investigate the factor structure of the O*NET-IP-SF (Rounds et al., 2010) using confirmatory factor analysis. Results demonstrated poor fit for the initial six-factor structure currently utilized by the instrument. Due to difficulties with total model fit, each domain was evaluated independently and modified to obtain improved fit. Several of these domains demonstrated problems with a homogeneous model, suggesting a greater variability than a single factor should produce. Using a data-driven approach to correcting these errors, problematic domains were restructured until their independent fit improved. Improvement in overall model fit was evident following the integration of those modified domains into a new O*NET-IP-SF model. Even these corrections did not produce a level of fit needed to satisfy traditional requirements for excellent fit. However, the changes did produce a final model (e.g., all six RIASEC domains) with indications of acceptable fit (e.g., CFI > .9 and RMSEA < .08; Hu & Bentler, 1999).
Taken together, fit observed in the O*NET-IP-SF domains showed structural improvements. These improvements carried over into the full form of the instrument and produced stronger evidence of validity. While these modifications did not achieve excellent fit using Hu and Bentler’s (1999) interpretation rules, the incorporated model changes did help to minimize the risk of Type I and Type II error (rejecting a model when it is true and accepting a model when it is not appropriate, respectively) during the interpretive use of the O*NET-IP-SF through improved measurement of the independent interest domains. Accordingly, this study provides a more valid psychometric form of the O*NET-IP-SF, despite some potential for model problems remaining unresolved. That the model did not produce the desired excellent levels of fit after several data-based corrections suggests several distinct challenges facing vocational assessment. Briefly summarized, results from this study suggest that (a) the current interpretive structure of the O*NET-IP-SF does not satisfy strict psychometric requirements, but modifications can be made to produce a stronger instrument; (b) thematic corrections within the domains suggest that there exist patterns of preference for specific interests within some RIASEC domains on the O*NET-IP-SF; and (c) elevations in interest scales may be associated with unintended word choice related to these aspects of each domain rather than the broad measurement of an intended interest area.
The current interpretive and scoring structure of the O*NET-IP-SF needs modification to produce desirable psychometric qualities (on both individual domains and the broader instrument). However, even with modifications, interpretation may still face some difficulties given that the instrument possesses psychometric qualities which remain unideal for an assessment tool. A portion of this error in assessment is likely due to the complex nature of vocational interests, with correlations between interest domains being expected to vary across different interest code types. For example, the I-A-S and I-A code types are similar but would produce increased model error due to the exclusion of the social domain in the latter. Not all individuals possessing high endorsements for investigative and artistic interests would necessarily also endorse the social interest domain. Distinct, but overlapping, code types such as these persist as a natural part of vocational assessment, and they help explain where some of the poorer fit originates. These expected differing relationships between the domains also provide a rationale for accepting fit statistics as appropriate which might otherwise be considered problematic. There remains simply more error associated with interest measurement because code type groupings overlap. The degree to which this error is deemed acceptable and appropriate remains yet to be defined; researchers in vocational psychology could strengthen their instruments by exploring this psychometric question.
However, other portions of error within the O*NET-IP-SF appear due to problems within the individual interest domains. This is where refinement is not only possible, but it is also needed. Each interest domain used within the O*NET-IP-SF should be able to measure a singular construct when evaluated independently. When problems appear within individual interest domains, it means that vocational assessment decisions may be based on fallacious assumptions, such as a belief that an elevation on the realistic interest category indicates a ubiquitous interest in the down-to-earth, physical, and hand- or tool-involved type of work associated with that domain. With corrected items in domains possessing medium to large correlations, it remains likely that scale elevations involving those items exist due to specific vocational preferences rather than broad, domain-level interests. Simply put, not all realistic work exists in the same form as all other realistic work. As vocational interest inventories improve their internal structure for each domain, they will increase the ability of career counselors to provide meaningful suggestions of interest fit.
Implications for Career Researchers
Vocational assessment instruments need further evaluation as to patterns of within-domain responses. Additionally, researchers may wish to take one of the two approaches in future assessment development. One approach emphasizes interest structures within each domain that accounts for different context and preferences for type of work (e.g., realistic domain may include facets comprising construction, machine operation, forestry, etc.). The second approach focuses on more general descriptions of job tasks that are thematic to the interest domain without providing details that may lead respondents to focus on contextual factors of where or how the job is conducted (e.g., enterprising focused on investing, managing, etc. as opposed to specifically “managing a retail store” or “market a new line of clothing”).
While the exact structure of the RIASEC remains in debate (e.g., Gati, 1991), interest inventories are typically examined using the six general occupational themes: RIASEC (Donnay et al., 2005). Commercial measures limit those who may access them and slow the improvement of measures and their underlying theory as well as contrast with Miller’s (1969) famous call. Public domain measures, like the O*NET IP-SF, do not possess that limitation and they are free and available to all. However, while the O*NET-IP-SF holds promise, it faces some difficulties in validity. Some of these problems persist due to the independent structural form of its interest domains. Others, however, exist because of the underlying problems facing current approaches to vocational interest assessment. As such, clinicians and researchers should be aware of shortcomings underlying the interest inventories they use and make research-based corrections to scoring and interpretation in order to enhance interpretive accuracy. Given the difficulties observed here in a college sample, further work with more diverse groups (ethnic, educational, etc.) should be done to ensure the structural form of the O*NET-IP-SF is able to demonstrate good psychometric properties and between group invariance. This investigation of invariance may not yet be possible as such testing requires an assumption of excellent model fit and this has yet to be established. Diverse groups stand likely to see particular benefit from improved psychometrics given how cultural values are centrally located within their vocational identity (Warlick, Ingram, Multon, & Vuyk, 2017).
Implications for Career Counselors
Human decision-making is mess management (Ackoff, 1974). Career assessment’s chief purpose remains to gather information from that mess that can be utilized in vocational decision-making (Whiston, 2013). Assessment interpretation stands as both an art and a science (MacCluskie, Welfel, & Towan, 2002). Clinicians should be aware of the psychometric limitations that appear within this instrument and remain likely to appear across other instruments taking a similar approach when assessing interests with clients. Counselors should seek clarification from the client during assessment interpretation about any thematic elements of work which emerge within the domains. Integrative batteries may offer a way to corroborate these thematic interests and offer a better sense of vocational needs and interests.
Despite issues with inventories, those who have interest inventories interpreted for them participate in more career exploration than those who do not (Randahl, Hansen, & Haverkamp, 1993). Despite its limitations, data from the O*NET-IP-SF can still be utilized to guide hypothesis formation (Hansen, 2013), to begin a narrative of manifest interests (Savickas, 2015), and to assess saliency of one’s cultural background and individual differences (Fouad & Kantamneni, 2008). The instrument offers a freely available tool grounded in current theory that may be useful to promoting engagement in a specific direction (Krieshok, Black, & McKay, 2009). The reactions, narratives, and patterns that the client identifies should be prized over the scale scores garnered from the instrument. The purpose of career assessment may be to gather and utilize information. However, as Krumboltz (2015) argues, the purpose of career counseling is “to launch the client on an exploratory task,” toward continuous occupational engagement not toward “making a permanent occupational choice.” Interests can facilitate this process.
There are several limitations to this study, which offer direction for future study. While participants from this study possess diverse elements, the ethnic composition is largely from two groups: Caucasian and African American. Although research has found no differences in model fit for RIASEC across ethnicity for those in the United States (Gupta, Tracey, & Gore, 2008), further confirmation of model fit for the O*NET-IP-SF should include other non-Anglo-American populations (e.g., those of Asian, Middle Eastern, Hispanic descent, etc.). Further assessment of the O*NET-IP-SF instrument utilizing a general population sample could provide an additional information about the influence of other factors which influence vocational interest development (e.g., education). Lastly, future analyses may benefit from using exploratory structural equation modeling (Asparouhov & Muthen, 2009), as it may provide a more relaxed interpretation regarding model fit. However, while not desirable, these issues are ubiquitous throughout the vocational literature. A strength of public domain instruments is that researchers can identify issues, make improvements, and ultimately, strengthen the instruments. Results from this study serve as a building block for research into the more accurate measurement of public domain vocational assessments.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
