Factor Structure and Gender Invariance of the Negative Outcome Expectations Scale Among Latiné Engineering College Students

Abstract

Using a sample of 883 Latiné engineering students, the present study explored the measurement characteristics of the Negative Outcome Expectations Scale for Engineers (NOES-E), applying advanced psychometric techniques, including bifactor and the exploratory structural equation modeling (ESEM) framework, to ascertain the scale’s most suitable factor structure. Additionally, measurement invariance across Latino and Latina participants was tested to investigate if the scale operates equally across gender groups and whether the latent mean scores are invariant. The results suggested that the bi-factor ESEM provided the best model fit when contrasted with other possible factor structures. The findings generally highlighted the general factor as reflecting the negative outcome expectations, suggesting that the use of total score is advised over the use of individual subscale scores. Findings from the measurement invariance testing across men and women indicated the NOES-E score can be compared between both groups.

Keywords

negative outcome expectations scale for engineers bifactor ESEM measurement invariance latiné engineering college students

Navigating the Factor Structure of the Negative Outcome Expectations Scale Among Latiné Engineering College Students

Latinés in the U.S. are driving both the growth and diversity of our country’s population and labor force (Fry et al., 2026). However, they remain underrepresented in STEM occupational fields which are favorably regarded due to their high earning potential, job stability and job growth. Engineering remains one of the most highly segregated occupational fields in the U.S. Although there has been an increase in the enrollment of Latinés in engineering programs and the awarding of engineering degrees to Latinés over the past decade, they remain underrepresented in the engineering workforce, comprising 9% of all engineers in the U.S. (SHPE-LDC, 2024). Further, the gendered racial disparities are striking. Latino men account for the vast majority of engineering degrees earned each year and of engineers in the labor force compared to Latina women (National Science Foundation, 2023), and Latino engineers earn more money than Latina engineers (SHPE-LDC, 2024). It is critical to understand those mechanisms that lead to and sustain engineering career selection and persistence among Latinés to better support and retain them in their engineering pursuits and to address longstanding inequity issues in their access to engineering careers.

Building upon Bandura’s (1996) social cognitive theory, social cognitive career theory (SCCT; Lent et al., 1999) proposes that outcome expectations—defined as the anticipated rewards stemming from one’s performance—serve as a crucial precursor influencing and directing subsequent career-related goal-directed behaviors. Per Bandura’s definition, outcome expectations include three core aspects: social (e.g., advantages to one’s family or social life), material or physical (e.g., financial gains), and self-evaluative (e.g., self-approval). A body of research documents the salient role that outcome expectations play in regards to career interests (e.g., Turner et al., 2019), career goal exploration (Lent et al., 2017), and proactive career behaviors (e.g., Korkmaz & Yam, 2023). Furthermore, in a meta-analysis of 143 SCCT STEM studies, Lent and colleagues (2018) reported that outcome expectations were significantly related to STEM interests and choice goals. In fact, outcome expectations exerted a stronger effect on STEM interests for persons of color relative to whites, and STEM choice goals for women and persons of color compared to their male and white counterparts, respectively. Thus, outcome expectations play a critical role in career processes and outcomes among individuals in STEM fields.

Prior studies on outcome expectations have mainly focused on its positive valence, namely the positive outcome expectations in career processes, overlooking the holistic picture of the outcome expectations (e.g., negative anticipations of the reward) that was initially conceptualized by Bandura. Additionally, negative outcome expectations were argued to be more influential than the positive anticipations in terms of its effect on career development process of individuals from marginalized groups, given the multiple systemic barriers that they face in the labor force (e.g., Fouad & Byars-Winston, 2005; Lent et al., 2000; Morrow et al., 1996). Specifically, for individuals from marginalized backgrounds, rewards may be less contingent upon the quality of their performance and merit and more on unfair external and systemic factors (Lent et al., 2000). Individuals from disenfranchised groups are more exposed to situations that involve negative outcome expectations (Fouad & Byars-Winston, 2005). Hence, underrepresented groups in engineering, such as Latiné college students face unique experiences and challenges with negative outcome expectations that may serve a critical role, and potentially an even stronger role than positive outcome expectations, in their career process and outcomes.

To address measurement gaps in the field, the Negative Outcome Expectations Scale in Engineering Scale (NOES-E; Lee et al., 2018) was created, to assess one’s degree of negative anticipation of rewards or outcomes in pursuing engineering. Due to the rising importance of engineering both nationally and globally (U.S. Bureau of Labor Statistics, 2024), much empirical attention has focused on recruiting and retaining exceptional students in engineering. Many of these studies include examining those constructs that may influence persistence intentions, such as negative outcome expectations.

The NOES-E is composed of four factors: Cultural-Related Stressors (CRS; “Limited access to mentors who understand me”), Personal Life and Work Balance (PLWB; “Not having time to maintain current friendships or being new ones”), Job Characteristics (JC; “Feeling frustrated with challenging tasks”), and Social Costs (SC; “Feeling like an outcast among potential romantic partners”). As can be seen from the sample items, the CRS factor evaluates the anticipation of adverse consequences linked to an unwelcoming work environment for people from marginalized backgrounds, particularly women and individuals of racially and ethnically diverse origins within the engineering field. PLWB measures expected challenges in balancing work and personal lives as members of engineering fields. JC assesses the apprehension and concerns regarding difficulties with highly demanding and rigorous tasks as engineers. SC measures the anticipated concerns of being excluded from the social circles, such as family, friends, and romantic partners, as a result of being an engineer. The four-factor structure of the scale encompasses various aspects of anticipated outcomes that are inclusive of the social, physical, and self-evaluative components that were initially defined in Bandura’s early works (Fouad & Guillen, 2006). Thus, the NOES-E represents the complex and multidimensional construct in alignment with Bandura’s (1996) initial conceptualization. The NOES-E was found to possess robust cross-sectional psychometric properties among the scale development sample of engineering students (Lee et al., 2018) as well as among a group of racially diverse college students (Suh et al., 2024). Additionally, the scale has been used among Latiné individuals in college settings to assess their expected negative career outcomes (Bonifacio et al., 2018; Flores et al., 2021a). These prior studies have demonstrated that the NOES-E scale is a valuable tool for gauging negative outcome expectations particularly among those from marginalized groups, including Latiné engineering college students.

Given its multidimensional nature, further investigation to better understand its factor structure is warranted. Additionally, approaches that allow flexibility in configuring the factor structure, such as exploratory structural equation model (ESEM), would ensure a more realistic and nuanced understanding of the latent structure (Asparouhov & Muthén, 2009; Prokofieva et al., 2023). Using confirmatory factor analysis, the original study established and identified a second-order factor structure as having the best fit to the data, consistent with the conceptualization of the NOE as comprising a general construct and four subfactors (Lee et al., 2018). However, the confirmatory factor analysis (CFA), a traditional approach for assessing dimensionality of scales, has limitations including rigidity and inflexibility for assuming independent item loading on only one latent factor (Su et al., 2019). This assumption is impractical and excessively strict for instruments measuring complex multidimensional constructs (Marsh et al., 2014), such as the NOES-E. Additionally, in the original study, the model fit of the second-order factor structure was identical to the correlated factor structure, indicating that the hierarchical ordering of the factor structure may not be superior to the correlated factor structure. Furthermore, hierarchical order models are difficult to interpret and challenging to find practical implications as the hierarchical order does not guarantee that the summation of all item scores is valid. The presence of a general factor, where all of the items are loaded onto one factor, would enable the aggregation of item scores, and its existence can be verified using a bi-factor model that distinguishes between a single global factor and multiple specific factors. Additionally, a general factor represents the broad construct of negative outcome expectations whereas specific factors represent the narrow and specific domains of the construct (Reise, 2012). Specific factors can still measure outcome expectations (even after accounting for the variance explained by the general factor) in a narrower scope pertaining to that specific construct (Gustafsson & Aberg-Bengtsson, 2010), which aligns with how the NOES-E is conceptualized.

In addition, although exploratory factor analysis (EFA) is believed to provide more precise estimates of latent factor correlations than CFA and thus is considered necessary for multidimensionality of the construct, EFA has been frequently questioned for being data-driven and unsuitable for confirmatory studies (Howard et al., 2018). Thus, Asparouhov and Muthén (2009) established Exploratory Structural Equation Modeling (ESEM) by combining the advantages of EFA and confirmatory factor analysis into an overarching SEM framework. Briefly, ESEM provides an appropriate model to investigate the source of multidimensionality due to the associations between indicators and the non-target construct in the form of cross-loadings (Su et al., 2019). Furthermore, ESEM yields better discriminant validity by providing more accurate and lower estimates of factor correlations (Asparouhov & Muthén, 2009). Given its strengths, ESEM has been widely adopted among studies that investigate the factor structure of psychological related constructs (e.g., Big Five measures; Marsh et al., 2012). At the same time, the bifactor ESEM (Morin et al., 2016) is a comprehensive and flexible nature of this approach, which accounts for both general and specific aspects of the component that closely reflect the construct’s multidimensionality. For instance, scholars (e.g., Howard et al., 2018; Morin et al., 2016) regard the bifactor ESEM as the promising method to describe complex dimensional structures, replacing the traditional CFA that is governed by strict assumptions. Given the multifaceted nature of the NOES-E (Lee et al., 2018), the current study explores if ESEM or the bifactor ESEM provides the best model fit. Furthermore, Hierarchical ESEM (H-ESEM), an extension of second order CFA, was also investigated for model comparison. H-ESEM conceptualizes that first order factors load onto the second order factor. H-ESEM is useful to determine if the scale reflects distinct multidimensionality rather than the presence of a dominant general factor. As the second order factor in H-ESEM influences items indirectly through the first order factor (e.g., Marsh et al., 2020), a summed score may not accurately represent the total score of the general factor.

Gender Differences in Experiences With Negative Outcome Expectations

SCCT highlights the effect of contextual factors on socio-cognitive career factors, such as career decision self-efficacy and career outcome expectations, that are considered salient in the overall career development process (Lent et al., 1999). Thus, the SCCT framework provides a nuanced understanding of how the career process, including the formation and manifestation of career constructs, may vary depending on one’s environment (Lent et al., 1994). Gender is one of the most frequently studied distal contextual factors within the SCCT framework and is believed to shape career outcome expectations indirectly through learning experiences (Lent et al., 2000). The longstanding and entrenched income disparity between men and women in the workplace, with Latina women in engineering earning only 58% of what white men in engineering earn in the U.S. (U.S. Bureau of Labor Statistics, 2024), creates an environment for Latina women to learn to anticipate and cultivate negative outcome expectations regarding their career. In other words, Latina women are continuously reminded that they will not be rewarded fully or equally for their performance relative to men. In addition, Latina women, especially in occupational fields dominated by men, frequently encounter messages that question their competence and their place within the field (e.g., Garriott et al., 2019). Such experiences likely lead to a greater level of negative outcome expectations for Latina women. In fact, numerous studies have shown that women of color engineers are confronted with notions that they are unappreciated, invisible, and undervalued (e.g., Garriott et al., 2019; Wilkins-Yel et al., 2019).

The question of whether there is a difference between Latino men and Latina women in terms of their NOES-E scores can be addressed only after it is known that the measurement operates the same across both groups (e.g., Chiorri et al., 2016). Thus, a comparison in NOES-E scores across Latino men and Latina women first requires measurement invariance testing. If there is measurement invariance, then the instrument would be deemed to operate the same across groups (Putnik & Bornstein, 2016) and score comparisons across groups can be considered valid. However, measurement invariance testing across both groups has not been conducted on NOES-E scores with the accurate factor structure. Thus, the current study aims to conduct measurement invariance testing across Latino men and Latina women after an accurate factor structure has been identified.

As mentioned above, invariance testing findings will ensure that the scale score can be compared across two groups. After obtaining the invariance findings across gender groups, studies that compare NOES-E scale scores may be helpful in understanding if men and women in engineering experience different levels of negative outcome expectations and whether this difference stems from latent construct level differences. Future studies may implement an intervention to target Latinés students’ negative outcome expectations and test if the intervention proved effective in reducing negative expectations for both groups or only for one of them. The measurement invariance findings merely informs us if the scale scores across groups can be compared, not whether there is a latent level true difference. The true difference in the construct can be identified from latent variance-covariance and latent mean comparisons across groups (i.e., true difference in the experiences with negative outcome expectations across two groups). Once the measurement invariance is established, the latent mean and latent variance-covariance invariance testing can be conducted (Barbosa-Leiker et al., 2013). Given the aforementioned findings that showcase Latina women engineers’ experiences with discrimination and low frequencies of rewards, it is hypothesized that there will be a difference in the latent mean level, with Latina women engineering college students showing a higher level of negative outcome expectations than Latino men engineering college students.

Present Study

In alignment with vocational psychology’s efforts to advance knowledge about the vocational experiences of marginalized workers (e.g., Blustein, 2013), the current study centers the vocational experiences of Latiné college students in an occupational major (and field) in which they are underrepresented. Further, to expand inclusive vocational psychology science and practice, we must develop tools that are validated with samples representing marginalized groups (Blustein et al., 2019; Flores et al., 2019, 2021b). Thus, the present study aimed to assess the factor structure of the NOES-E among a group of Latiné engineering college students. To do so, we conduct a comparative analysis of factor structures with the hypothesis that bifactor Exploratory Structural Equation Modeling (ESEM) would yield the best factor structure, outperforming both second-order CFA (identified in the original study; Lee et al., 2018), ESEM, and hierarchical ESEM. Additionally, based on the established factor structure, we conduct measurement invariance testing across men and women participants to investigate if the scale scores operate equally across the groups. Once the invariance is established, we examine if negative outcome expectations levels are truly different between these groups by testing for latent mean and variance-covariance invariance testing. We expect that women engineering college students would show a higher level of negative outcome expectations than men.

Method

Participants

A total of 883 Latiné engineering college students were included in the study. In terms of gender, 523 identified as men (59.2%), 359 identified as women (40.7%), and one identified as transgender (0.1%). The mean age of the participants was 21.4 years (SD = 3.5, range = 18–50). As for the generational status, the majority identified as second generation (n = 424, 48.1%), followed by first generation (n = 287, 32.5%), fourth generation (n = 63, 7.1%), third generation (n = 62, 7.0%), and fifth generation (n = 46, 5.2%). One person did not report their generational status. Most of the participants were juniors (n = 265, 30.0%), followed by seniors (n = 231, 26.2%), sophomores (n = 196, 22.2%), and first-year students (n = 184, 20.8%). Seven students identified as other (0.8%), which included transfer students or completing a second major in engineering, and did not consider themselves to fit in the categories above. Participants’ average GPA was 3.19 on a 4.0 scale (range = 0.0–4.0)

Procedure

The present study draws on data from a larger 5-year longitudinal project examining how contextual, cultural, and social cognitive factors shape Latiné undergraduate students’ academic engagement, satisfaction, and persistence. Using multiple methodological approaches (e.g., longitudinal quantitative and qualitative methods), the larger project aims to evaluate SCCT’s generalizability to Latinés and across gender as well as to explore potential theoretical refinements. An additional goal is to validate measures relevant to engineering academic and career development, such as the NOES-E (the focus of the current study), with Latiné engineering undergraduates.

The data used for the current study are drawn from the first of a five-wave annual longitudinal quantitative survey study. Participants were recruited from 11 partnering institutions selected using Department of Education data that identified the top 40 U.S. higher education institutions awarding the highest number of undergraduate engineering degrees to Latiné students. These partners included both Predominately White Institutions (PWIs) and Hispanic-Serving Institutions (HSIs) and were geographically distributed across the northeastern, southeastern, southern, southwestern, and western regions of the United States. . Initial participant recruitment included multiple strategies. That is, faculty and student leaders at each institution recruited participants through announcements and flyers distributed in undergraduate engineering courses, student organization meetings, and high-traffic areas within college buildings, as well as via email invitations sent through college-wide and student organization listservs. Reminder emails were sent at two-week intervals until recruitment goals were achieved. All recruitment materials included a brief study description and a URL or QR code linking to a Qualtrics survey that included the informed consent information and study measures. Participants received a $20 gift card for their participation in this first wave of data collection. Institutional approval was obtained and treatment of participants was conducted according to the ethical guidelines and approved protocol.

Measurement

The Negative Outcome Expectations Scale for Engineering (NOES-E; Lee et al., 2018) was used to assess Latiné engineers’ experiences with adverse outcomes in their career in engineering fields. The scale is comprised of four factors: Cultural-Related Stressors (“Limited access to mentors who understand me”), Personal Life and Work Balance (“Not having time to maintain current friendships or being new ones”), Job Characteristics (“Feeling frustrated with challenging tasks”), and Social Costs (“Feeling like an outcast among potential romantic partners”). The scale was found to have strong psychometric properties from the Lee et al.’s original study, where the internal consistency ranged from .80 to .90 across the subscales and .94 for the total 21 items. Additionally, the total score showed negative significant associations with related constructs (e.g., engineering-intended persistence, engineering positive outcome expectations), showcasing evidence for validity (Lee et al., 2018). The current study’s reliability calculated with the omega coefficient ranged .67 to .83 for the subscales and .92 for the total items (see Table 2).

Analysis Plan

To address the research question, the traditional CFA with second-order model (as in the original study), ESEM, H-ESEM, and bifactor ESEM were configured and their respective model fit indices were contrasted. In the traditional CFA model, items were only allowed to load onto one of the four factors identified in the original study and no cross-loadings on other factors were allowed. In the ESEM model, all cross-loadings were freely estimated while the estimation was “targeted” to be close to zero using the target rotation (Morin et al., 2016). The bifactor ESEM was estimated under the classic bifactor assumptions, in which all items were allowed to concurrently define a general factor (G-factor) as well as the four specific factors (S-factor). Given the non-nested nature of these models, comparisons were conducted based on the lowest AIC, BIC, and adjusted BIC as having the best fit (Vrieze, 2012). The metrics such as Comparative Fit Index (CFI) and Tucker-Lewis Index (TLI), Root Mean Square Error of Approximation (RMSEA), and Standardized Root Mean Square Residual (SRMR) were used to evaluate the model: CFI, TLI > .95 were considered to be good fit and over .90 as acceptable fit, RMSEA < .05 was considered a good fit and between .06 and .08 as acceptable fit, SRMR < .08 was considered a good fit and < .10 as acceptable fit (Hu & Bentler, 1999).

The measurement invariance was tested to see if there was observed invariance, which indicates that an instrument assesses the construct in the same manner for different groups. To determine the invariance of NOES-E scores across men and women, the magnitude of the changes in the model fit indices were examined. Non-significant results in the chi-square difference test indicates support for the presence of invariance (i.e., two nested models are not significantly different from each other), but this test typically is not used as the primary criterion for determining invariance because of the decreased robustness due to its sensitivity to sample size. Hence, the benchmarks used to determine the invariance in this study were $Δ$ CFI less than .01 and $Δ$ RMSEA less than .015 (Cheung & Rensvold, 2002; Marsh et al., 2010).

Results

Prior to testing the hypotheses, a series of preliminary analyses were conducted. Missing data was insubstantial from the covariance coverage exceeding .99. The missing pattern was not completely at random from Little’s missing at completely random test (Chi-square = 200.875, df = 123, p = .000); but it was assumed missing at random (MAR) as the missingness was significantly associated with other observed variables (Schlomer et al., 2010). Specifically, college GPA and college major GPA were significantly related to the missing and thus the missing was assumed at random (MAR). Given the MAR, the full information maximum likelihood (FIML) approach (with the observed variables as auxiliary) was used to manage the missing values, which is found to perform well under MAR condition (Schlomer et al., 2010). The item-level descriptive statistics, such as the mean, standard deviation, and inter-item correlation, is available in the supplementary table. Data analyses were performed using Mplus 8.1 (2012–2022).

Factor Structure of the NOES-E

To investigate if the bifactor ESEM would have the best fit to the data among other models, corresponding to the hypothesis, the model comparison on the original second-order CFA factor, ESEM, and the bifactor ESEM was conducted. In line with the hypothesis, from the AIC, BIC, and adjusted BIC values, the results indicated that the bifactor ESEM had the best model fit over the second-order CFA factor and ESEM (see Table 1). Additionally, the bifactor ESEM model was evaluated to have an excellent model fit in reference to the benchmarks identified above. Furthermore, to ensure that the factor structure does closely reflect a bi-factor ESEM structure, the percent of uncontaminated correlations (PUC; proportion of correlations among items, influenced by the general factor and minimally influenced from specific factors) and Explained Common Variance (ECV; proportion of common variance attributable to the general factor) was calculated, in which the values were .79 and .73, respectively. This suggests a strong indication of a unidimensional or general factor of the scale (Rodriguez et al., 2016). For instance, 73% of the common variance among the items was explained by the general negative career outcome expectations factor. The general factor was well-defined in the factor solution as well for its largely moderate to strong item loadings (

λ

= .33 - .65, M_λ = .54) (see Table 2). Furthermore, the omega hierarchical (

ω

H)—an index reflecting the proportion of reliable systemic variance of a subscale score after partitioning out the variability attributed to the general factor (Reise et al., 2013, p. 225)—of each subscale was all less than .3, indicating a weak reliability on their own after the variance of the general factor was removed. The

ω

H of the general factor was .87, indicating a high level of internal reliability. Overall, the NOES-E’s bi-factor ESEM structure was found to have the best fit to the data with the general factor having a substantial variance among the items, indicating better use of overall scale score over the independent subscale scores.

Table 1.

Fit Indices for the Second Order CFA, ESEM, and Bifactor ESEM of the NOES-E

Model	AIC	BIC	adj. BIC	$χ$ ² (df)	CFI	TLI	RMSEA 90% CI [LL, UL]	SRMR
A	76825.39	77119.08	76922.19	1189.10* (190)	.826	.808	.079 [.075, .083]	.085
B	76097.01	76665.45	76284.37	344.72* (132)	.963	.941	.044 [.038, .049]	.024
C	76109.50	76656.99	76288.61	363.21* (136)	.961	.939	.045 [.039, .050]	.027
D	76035.00	76679.23	76247.34	250.71* (116)	.977	.958	.037 [.031, .043]	.020

Note. A = Second Order CFA; B = ESEM; C = ; Hierarchical ESEM; D = Bifactor ESEM; AIC = Akaike Information Criterion; BIC = Bayesian Information Criterion; CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root Mean Square Error of Approximation; SRMR = Standardized Root Mean Square Residual; *p < .001.

Table 2.

Standardized Factor Loadings for the Bifactor ESEM of the NOES-E

Item	General factor	PLWB	JC	CRS	SC
PLWB_1	.46	.04	−.08	.00	.10
PLWB_2	.49	.09	−.18	−.03	−.09
PLWB_3	.57	.25	−.12	.01	.06
PLWB_4	.49	.19	−.09	−.10	−.04
PLWB_5	.47	.42	.07	.09	.00
JC_1	.63	−.10	−.02	−.06	.02
JC_2	.33	.01	−.32	−.05	−.10
JC_3	.44	.22	.08	.07	−.06
JC_4	.60	−.08	.25	.11	.00
JC_5	.57	−.07	.36	−.06	−.09
CRS_1	.62	−.13	−.13	−.01	−.08
CRS_2	.56	.12	−.01	.05	.28
CRS_3	.56	.14	.06	.21	−.02
CRS_4	.49	−.02	−.09	.89	−.01
CRS_5	.64	.03	.21	.05	.00
CRS_6	.52	−.12	.19	.24	−.04
SC_1	.61	−.10	−.07	.01	.14
SC_2	.54	.15	.05	−.03	.29
SC_3	.65	−.09	−.06	.00	.62
SC_4	.60	.00	.00	.01	.58
SC_5	.57	.13	.22	.02	.15
ECV	.73	.03	.04	.11	.10
$ω$	.92	.67	.67	.81	.83
$ω$ H	.87	.09	.01	.12	.22

Note. PLWB = Personal Life and Work Balance; JC = Job Characteristics; CRS = Culture-related Stressors; SC = Social Costs; ECV = explained common variance; $ω$ = omega coefficient; $ω H$ = omega hierarchical.

Measurement Invariance Testing Across Men and Women

After the factor structure of the NOES-E was established, we proceeded with invariance testing of the scale across men and women. As seen in Table 3, the scale featuring the bifactor ESEM factor structure demonstrated invariance up to the scalar level, as the model fit deterioration did not exceed the benchmark established above. The residual invariance was shown to have a significant drop in the model fit, so the residuals that significantly differed (d = .2, Cohen, 1992) were identified and released. Four items were identified: “challenges balancing between work responsibilities and family obligations,” “worrying about being perceived negatively because of plans to have family or already having a family,” “feeling like an outcast among family and friends,” and “being less attractive to friends.” Specifically, Latina found to score higher in residual variances than Latino on these items. The model fit of the partial residual model did not drop substantially from the scalar invariance and thus, we proceeded to the next step.

Table 3.

Model Invariance Testing Across Gender

	$χ$ ²	$Δ$ $χ$ ²	df	$Δ$ df	CFI	$Δ$ CFI	TLI	$Δ$ TLI	RMSEA	$Δ$ RMSEA	SRMR	$Δ$ SRMR
Configural	392.85		231		.973		.950		.041		.025
Metric	526.15	133.30*	321	90	.964	−.009	.951	.001	.040	−.001	.035	.010
Scalar	571.93	45.78**	328	7	.959	−.005	.947	−.004	.042	.002	.036	.001
Residual	693.36	121.43**	348	20	.942	−.017	.930	−.017	.049	.007	.046	.010
Partial residual	634.45	62.52**	344	16	.951	−.008	.940	−.007	.045	.003	.041	.005
Latent variance	719.28	84.83**	359	15	.939	−.012	.929	−.011	.049	.004	.060	.019
Latent mean	827.08	107.80**	364	5	.922	−.017	.910	−.019	.055	.006	.062	.002

Note. CFI = Comparative Fit Index; TLI = Tucker-Lewis Index; RMSEA = Root Mean Square Error of Approximation; SRMR = Standardized Root Mean Square Residual; *p < .01, **p < .001.

Upon the measurement invariance findings, the latent mean invariance was conducted to examine if there is a true difference in the construct of negative outcome expectations between men and women. As seen in Table 3, the model fit substantially dropped from the residual level, indicating a difference in the true construct level across both groups. The standardized latent mean difference was computed using Cohen’s (1988) d to compute the standardized latent mean difference with the latent mean and latent variance obtained from both men and women (e.g., Little, 1997). The difference in the general factor was less than small and insubstantial (d = .1). However, the Personal Life and Work Balance (PLWB; d = .2) and Cultural-Related Stressors (CRS; d = .6) factors were shown to be different between the two groups. Specifically, women engineering students showed a higher level in those two latent factors. Interestingly, Social Cost (SC; d = .2) was higher among men engineering students than women students. Thus, the findings partially supported the hypothesis that women engineering students may experience greater levels of negative outcome expectations, specifically concerning their outlook on personal and work life balance and being rewarded unfairly due to one’s identity. Men engineering students were found to have stronger negative perceptions of being accepted and included among family, friends, and romantic partners, due to their commitment to the engineering field. However, given the low omega hierarchical ( $ω$ H) of each specific factor, findings that show such differences require caution in interpretation.

Discussion

The current study investigated the structural characteristics of the NOES-E, a scale that assesses negative career outlook among those in engineering fields. The original study suggested a second factor structure of the scale, supporting the use of total scale score. However, the model fit of the second factor structure was indistinguishable from the correlated factor structure in the original study, raising questions as to whether the established second factor structure may reflect an accurate factor structure of the scale. Additionally, the adoption of rigid and inflexible confirmatory factor analysis in the original study was identified as a limitation. Thus, the dimensionality of the NOES-E scale was assessed by employing the more flexible ESEM approach with particular focus on the bifactor ESEM due to its established advantage over alternative approaches in exploring a more realistic factor structure. After the factor structure was identified, we tested for measurement invariance of the scale across men and women and examined if there are differences in the latent mean regarding negative outcome expectations. Investigations on whether the scale operates equally across gender groups, which needs to be established first to compare the scale scores, have not been investigated. Therefore, the current study findings were deemed to contribute to the existing literature.

Corresponding to the first hypothesis, bifactor ESEM had the best fit to the data. Complimentary metrics, such as PUC and ECV strongly suggested a general factor of the scale, in which the findings indicate the use of total score (rather than the reliance on each individual subscale) may be most meaningful and valid in terms of the representation of the negative outcome expectations construct. Additionally, given the relatively low omega hierarchical ( $ω$ H) of each specific factor, it was deemed less reliable to use each separate subscales and their scores. Thus, future uses of the NOES-E should prioritize the total scale score in research and practice. Such a finding is not uncommon among studies that investigate psychological constructs, for which the unique contribution of each subscale considerably diminishes once the general factor is accounted for (e.g., Lee & Kim, 2024).

In this study, we also investigated the measurement invariance of the NOES-E scale across men and women to see if the scale operates equally and whether there is a latent level difference in the experiences with negative outcome expectations across the women and men Latiné engineering students. The findings showcased the invariance across the group in the NOES-E’s factor loadings, item intercepts and residuals–although women exhibited slightly higher residual variances than men on some items, potentially reflecting gender-specific experiences regarding negative outcome expectations (e.g., challenges in balancing work and family). The scale’s overall invariant operation means that future studies can compare NOES-E scores between Latiné men and women.

Additionally, there were some differences in the latent variance-covariance and latent mean between Latino men and Latina women, where women engineering students were found to have greater levels of experiences in two factors: Personal Life and Work Balance and Cultural-Related Stressors. These differences may reflect that women engineering college students experience a significantly higher level of negative outcome expectations than men in these aspects, particularly regarding the demands to balance family and work as well as dealing with hostile environments in the field. However, given that the use of subscales are not recommended for use due to the low omega hierarchical ( $ω$ H; i.e., low reliability), the remaining components of specific factors and their score differences across genders may not be as meaningful and the total scale is considered to function invariantly across two genders.

Implications

The scale demonstrated sound psychometric properties in measuring the negative outcome expectations. Given the significant underrepresentation of Latiné engineers (Fry et al., 2026; Society of Hispanic Professional Engineers-Latino Donor Collaborative, 2024) and the critical role that negative outcome expectations play in the career development process (Fouad & Byars-Winston, 2005; Lent et al., 2019), this scale’s support for a general factor, which gives a snapshot of experiences with negative outcome expectations, facilitates an efficient understanding of their experiences within the engineering field. Furthermore, the finding of measurement invariance across binary genders suggests that the scale functions equivalently in measuring negative outcome expectations for both Latina and Latino individuals, offering significant implications for practice and research. For instance, vocational counselors and psychologists interested in the effectiveness of interventions aimed at reducing negative outcome expectations among Latiné engineering students (e.g., inclusive policy changes or mentorship programs) could test whether these interventions are effective across genders. Similarly, examining the impact of environmental factors within the educational setting (e.g., classroom gender ratios) on differing levels of negative outcome expectations experiences across genders could help identify targets for academic setting change to improve Latina engineering students’ adjustment to the field.

From a research perspective, future studies can confidently aggregate data for Latiné individuals, as the scale’s consistent operation across groups allows this practice and the larger sample size enhances statistical power. Finally, although minor statistical differences were identified (e.g., in residuals and latent means on a couple specific factors), the presence of a dominant general factor suggests that these results showcasing non-invariance across genders should be interpreted with caution. It is interesting that the scale showed invariance across two genders even with different experiences within engineering (e.g., Garriott et al., 2019; Wilkins-Yel et al., 2019). Given the invariance observed in the broad general factor in this study, the invariance finding may align with studies on common cognitive processes (Fiske & Taylor, 2020). Specifically, experiences with negative outcome expectations may differ across both genders, but Latina and Latino students likely interpreted and responded to the items in a similar manner, resulting in an invariance finding. At the same time, this could indicate that the items may not fully capture the nuanced experiences given that some research highlights differences in gendered experiences (e.g., Garriott et al., 2019). Items and factors could be examined in future qualitative studies to better understand their experiences.

Limitations and Future Direction

There are several limitations of this study worth noting along with future directions to address the issue. First, given the sample was limited to Latiné engineering college students, applying these findings to other samples and contexts requires caution (e.g., Latiné engineers). Future studies with diverse samples in varied academic and work contexts, such as Latiné engineers in the field, is warranted to verify these findings. Second, though the bifactor ESEM was shown to have measurement invariance, four items were detected to be not invariant in the residual level. Future studies may replicate the model, with and without the items, to see if what we observed in this study is replicable to ensure it is not a spurious finding. Third, the study only included one-time point data. With the identified factor structure (i.e., bifactor ESEM), future studies may consider conducting measurement invariance testing across time (i.e., longitudinal measurement invariance testing). Findings would enable us to make comparisons pre- and post-intervention or policy implementation and see if interventions or implementations of policies are effective. Similarly, only binary gender groups were included as a factor for potential invariance; however, other factors such as work status (i.e., student vs. graduated and employed) can be tested for its invariance to make reliable and valid conclusions regarding differences in the scores across work statuses.

Conclusion

This study investigated the psychometric properties of the Negative Outcome Expectations Scale for Engineers (NOES-E) with a sample of Latiné engineering students, contributing to the cultural validity of this measure. Advanced psychometric techniques, specifically bifactor and exploratory structural equation modeling (ESEM), were employed to determine the most appropriate factor structure. The study also examined measurement invariance across gender to ensure the scale functions consistently for Latino and Latina students. Results indicated that the bifactor ESEM model provided the best fit compared to alternative factor structures. These findings suggest that the general factor should be considered, with the total score preferred over subscale scores. Finally, the invariance results confirm that NOES-E scores are comparable across gender groups. The findings that support the scale’s robust psychometric properties imply the scale can be widely used to assess negative outcome expectations for practice and research purposes. Additionally, the scale’s support for the dominant general factor provides an efficient snapshot of the negative outcome expectations experiences by using the total score.

Supplemental Material

Supplemental Material - Factor Structure and Gender Invariance of the Negative Outcome Expectations Scale Among Latiné Engineering College Students

Supplemental Material for Factor Structure and Gender Invariance of the Negative Outcome Expectations Scale Among Latiné Engineering College Students by Han Na Suh, Lisa Y. Flores and Rachel Navarro in Journal of Career Assessment.

Footnotes

ORCID iD

Han Na Suh

Ethical Considerations

This study was approved by the institutional review board at University of Missouri-Columbia. Additionally, this study adhered to the ethical standards outlined in the Declaration of Helsinki.

Author Contributions

The first author designed the study, performed the data analysis, and led the manuscript writing process. The second author secured the funding, collected the data, and edited the work. The third author secured the funding and collected the data, and edited the work.

Funding

The author disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the data collection was funded by the National Science Foundation EHR-2000607/2000636 and National Science Foundation DUE-1430614/1430640.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data Availability Statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.*

Supplemental Material

Supplemental material for this article is available online.

References

Asparouhov

Muthén

(2009). Exploratory structural equation modeling. Structural Equation Modeling: A Multidisciplinary Journal, 16(3), 397–438. https://doi.org/10.1080/10705510903008204

Bandura

(1996) Self-efficacy: The exercise of control (Vol. 11). Freeman.

Barbosa‐Leiker

Kostick

Lei

McPherson

Roper

Hoekstra

Wright

(2013). Measurement invariance of the perceived stress scale and latent mean differences across gender and time. Stress and Health, 29(3), 253–260. https://doi.org/10.1002/smi.2463

Blustein

(2013). The psychology of working: A new perspective for career development, counseling, and public policy. Routledge.

Blustein

D. L.

Ali

S. R.

Flores

L. Y.

(2019). Vocational psychology: Expanding the vision and enhancing the impact. The Counseling Psychologist, 47(2), 166–221. https://doi.org/10.1177/0011000019861213

Bonifacio

Gushue

G. V.

Mejia-Smith

B. X.

(2018). Microaggressions and ethnic identity in the career development of Latina college students. The Counseling Psychologist, 46(4), 505–529. https://doi.org/10.1177/0011000018776909

Cheung

G. W.

Rensvold

R. B.

(2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), 233–255. https://doi.org/10.1207/S15328007SEM0902_5

Chiorri

Marsh

H. W.

Ubbiali

Donati

(2016). Testing the factor structure and measurement invariance across gender of the big five inventory through exploratory structural equation modeling. Journal of Personality Assessment, 98(1), 88–99. https://doi.org/10.1080/00223891.2015.1035381

Cohen

(1988). Statistical power analysis for the behavioral sciences (2nd ed.). Routledge. https://doi.org/10.4324/9780203771587

10.

Cohen

(1992). A power primer. Psychological Bulletin, 112(1), 155–159. https://doi.org/10.1037/0033-2909.112.1.155

11.

Fiske

S. T. T.

Taylor

S. E.

(2020). Social cognition: From brains to culture. Sage.

12.

Flores

L. Y.

Martinez

L. D.

McGillen

G. G.

Milord

(2019). Something old and something new: Future directions in vocational research with people of color in the United States. Journal of Career Assessment, 27(2), 187–208. https://doi.org/10.1177/1069072718822461

13.

Flores

L. Y.

Navarro

R. L.

Lee

B. H.

Diaz

Martinez

(2021a). Social cognitive predictors of latinx and white engineering students’ academic satisfaction and persistence intentions: Exploring interactions among social identities and institutional context. Journal of Vocational Behavior, 127, 103580. https://doi.org/10.1016/j.jvb.2021.103580

14.

Flores

L. Y.

Settles

McGillen

G. G.

Davis

T. M.

(2021b). Critical contributions to scholarship on women and work: Celebrating 50 years of progress and looking ahead to a new decade. Journal of Vocational Behavior, 126, 103490. https://doi.org/10.1016/j.jvb.2020.103490

15.

Fouad

N. A.

Byars-Winston

(2005). Cultural context of career choice: Meta-analyses of race/ethnicity differences. The Career Development Quarterly, 53(3), 223–233. https://doi.org/10.1002/j.2161-0045.2005.tb00992.x

16.

Fouad

N. A.

Guillén

(2006). Outcome expectations: Looking to the past and potential future. Journal of Career Assessment, 14(1), 130–142. https://doi.org/10.1177/1069072705281370

17.

Fry

Passel

Hays

(2026). The United States at 250: How the country has changed in the past 50 years. Pew Research Center. https://www.pewresearch.org/social-trends/2026/03/25/the-united-states-at-250-how-the-country-has-changed-in-the-past-50-years/#demographics.

18.

Garriott

P. O.

Navarro

R. L.

Flores

L. Y.

Lee

H. S.

Carrero Pinedo

Slivensky

Atilano

Lin

C. L.

Gonzalez

Luna

Lee

B. H.

(2019). Surviving and thriving: Voices of Latina/o engineering students at a Hispanic serving institution. Journal of Counseling Psychology, 66(4), 437–448. https://doi.org/10.1037/cou0000351

19.

Gustafsson

J. E.

Aberg-Bengtsson

(2010). Unidimensionality and interpretability of psychological instruments. In Embretson

S. E.

(Ed.), Measuring psychological constructs: Advances in model-based approaches (pp. 97–121). American Psychological Association.

20.

Howard

J. L.

Gagné

Morin

A. J.

Forest

(2018). Using bifactor exploratory structural equation modeling to test for a continuum structure of motivation. Journal of Management, 44(7), 2638–2664. https://doi.org/10.1177/0149206316645653

21.

L. T.

Bentler

P. M.

(1999). Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Structural Equation Modeling: A Multidisciplinary Journal, 6(1), 1–55. https://doi.org/10.1080/10705519909540118

22.

Korkmaz

Yam

F. C.

(2023). What role do visions about future and vocational outcome expectations have on proactive career behaviors? Examining a mediation model on university students. Educational Policy Analysis and Strategic Research, 18(2), 86–106. https://doi.org/10.29329/epasr.2023.548.5

23.

Lee

Kim

Y. E.

(2024). A comparison of factor structure models for the mental health continuum-short form (MHC-SF) among preschool teachers in South Korea. Current Psychology, 43(5), 4678–4687. https://doi.org/10.1007/s12144-023-04683-1

24.

Lee

H. S.

Flores

L. Y.

Navarro

R. L.

Suh

H. N.

(2018). Development and validation of the negative outcome expectations scale in engineering (NOES-E). Journal of Career Assessment, 26(1), 52–67. https://doi.org/10.1177/1069072716679923

25.

Lent

R. W.

Brown

S. D.

Hackett

(1994). Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior, 45(1), 79–122. https://doi.org/10.1006/jvbe.1994.1027

26.

Lent

R. W.

Brown

S. D.

Hackett

(2000). Contextual supports and barriers to career choice: A social cognitive analysis. Journal of Counseling Psychology, 47(1), 36–49. https://doi.org/10.1037/0022-0167.47.1.36

27.

Lent

R. W.

Hackett

Brown

S. D.

(1999). A social cognitive view of school-to-work transition. The Career Development Quarterly, 47(4), 297–311. https://doi.org/10.1002/j.2161-0045.1999.tb00739.x

28.

Lent

R. W.

Ireland

G. W.

Penn

L. T.

Morris

T. R.

Sappington

(2017). Sources of self-efficacy and outcome expectations for career exploration and decision-making: A test of the social cognitive model of career self-management. Journal of Vocational Behavior, 99, 107–117. https://doi.org/10.1016/j.jvb.2017.01.002

29.

Lent

R. W.

Morris

T. R.

Penn

L. T.

Ireland

G. W.

(2019). Social–cognitive predictors of career exploration and decision-making: Longitudinal test of the career self-management model. Journal of Counseling Psychology, 66(2), 184–194. https://doi.org/10.1037/cou0000307

30.

Lent

R. W.

Sheu

H.-B.

Miller

M. J.

Cusick

M. E.

Penn

L. T.

Truong

N. N.

(2018). Predictors of science, technology, engineering, and mathematics choice options: A meta-analytic path analysis of the social–cognitive choice model by gender and race/ethnicity. Journal of Counseling Psychology, 65(1), 17–35. https://doi.org/10.1037/cou0000243

31.

Little

T. D.

(1997). Mean and covariance structures (MACS) analyses of cross-cultural data: Practical and theoretical issues. Multivariate Behavioral Research, 32(1), 53–76. https://doi.org/10.1207/s15327906mbr3201_3

32.

Marsh

H. W.

Guo

Dicke

Parker

P. D.

Craven

R. G.

(2020). Confirmatory factor analysis (CFA), exploratory structural equation modeling (ESEM), and set-ESEM: Optimal balance between goodness of fit and parsimony. Multivariate Behavioral Research, 55(1), 102–119. https://doi.org/10.1080/00273171.2019.1602503

33.

Marsh

H. W.

Lüdtke

Muthén

Asparouhov

Morin

A. J. S.

Trautwein

Nagengast

(2010). A new look at the big five factor structure through exploratory structural equation modeling. Psychological Assessment, 22(3), 471–491. https://doi.org/10.1037/a0019227

34.

Marsh

H. W.

Morin

A. J. S.

Parker

P. D.

Kaur

(2014). Exploratory structural equation modeling: An integration of the best features of exploratory and confirmatory factor analysis. Annual Review of Clinical Psychology, 10(1), 85–110. https://doi.org/10.1146/annurev-clinpsy-032813-153700

35.

Marsh

H. W.

Nagengast

Morin

A. J.

(2012). Measurement invariance of big-five factors over the life span: ESEM tests of gender, age, plasticity, maturity, and la dolce vita effects. Developmental Psychology, 49(6), 1194–1218. https://doi.org/10.1037/a0026913

36.

Morin

A. J.

Arens

A. K.

Tran

Caci

(2016). Exploring sources of construct‐relevant multidimensionality in psychiatric measurement: A tutorial and illustration using the composite scale of morningness. International Journal of Methods in Psychiatric Research, 25(4), 277–288. https://doi.org/10.1002/mpr.1485

37.

Morrow

S. L.

Gore

P. A.

Campbell

B. W.

(1996). The application of a sociocognitive framework to the career development of lesbian women and gay men. Journal of Vocational Behavior, 48(2), 136–148. https://doi.org/10.1006/jvbe.1996.0014

38.

National Science Foundation . (2023). Diversity and STEM: Women, Minorities, and Persons with Disabilities, 2023 (Report No. In NSF (23-315). National Center for Science and Engineering Statistics. https://nsf-gov-resources.nsf.gov/doc_library/nsf23315-report.pdf?VersionId=OfErRcu.MTEp_KCzcuAfyfIkh0cU_XQO.

39.

Prokofieva

Zarate

Parker

Palikara

Stavropoulos

(2023). Exploratory structural equation modeling: A streamlined step by step approach using the R project software. BMC Psychiatry, 23(1), 546. https://doi.org/10.1186/s12888-023-05028-9

40.

Putnick

D. L.

Bornstein

M. H.

(2016). Measurement invariance conventions and reporting: The state of the art and future directions for psychological research. Developmental Review, 41, 71–90. https://doi.org/10.1016/j.dr.2016.06.004

41.

Reise

S. P.

(2012). The rediscovery of bifactor measurement models. Multivariate Behavioral Research, 47(5), 667–696. https://doi.org/10.1080/00273171.2012.715555

42.

Reise

S. P.

Bonifay

W. E.

Haviland

M. G.

(2013). Scoring and modeling psychological measures in the presence of multidimensionality. Journal of Personality Assessment, 95(2), 129–140. https://doi.org/10.1080/00223891.2012.725437

43.

Rodriguez

Reise

S. P.

Haviland

M. G.

(2016). Evaluating bifactor models: Calculating and interpreting statistical indices. Psychological Methods, 21(2), 137–150. https://doi.org/10.1037/met0000045

44.

Schlomer

G. L.

Bauman

Card

N. A.

(2010). Best practices for missing data management in counseling psychology. Journal of Counseling Psychology, 57(1), 1–10. https://doi.org/10.1037/a0018082

45.

Society of Hispanic Professional Engineers-Latino Donor

Collaborative.

(2024). The 2024 SHPE-LDCC U.S. Latinos in engineering and tech report . https://shpe.org/wp-content/uploads/2025/02/2024-SHPE-LDC-U.S.-Latinos-in-Engineering-and-Tech-Report-Final.pdf

46.

Tay

Liao

H. Y.

Zhang

Rounds

(2019). Toward a dimensional model of vocational interests. Journal of Applied Psychology, 104(5), 690–714. https://doi.org/10.1037/apl0000373

47.

Suh

H. N.

Flores

L. Y.

Johanson-Murray

Park

Lee

F. Y.

(2024). Invisibility and negative career outcome expectations among racially and ethnically diverse college students at a predominantly white institution. Journal of Career Assessment, 32(1), 63–78. https://doi.org/10.1177/10690727231173729

48.

Turner

S. L.

Joeng

J. R.

Sims

M. D.

Dade

S. N.

Reid

M. F.

(2019). SES, gender, and STEM career interests, goals, and actions: A test of SCCT. Journal of Career Assessment, 27(1), 134–150. https://doi.org/10.1177/1069072717748665

49.

U.S. Bureau of Labor Statistics . (2024). Highlights of women's earnings in 2023 (Report No. 1111). U.S. Department of Labor. https://www.bls.gov/opub/reports/womens-earnings/2023/

50.

Vrieze

S. I.

(2012). Model selection and psychological theory: A discussion of the differences between the Akaike information criterion (AIC) and the Bayesian information criterion (BIC). Psychological Methods, 17(2), 228–243. https://doi.org/10.1037/a0027127

51.

Wilkins-Yel

K. G.

Hyman

Zounlome

N. O. O.

(2019). Linking intersectional invisibility and hypervisibility to experiences of microaggressions among graduate women of color in STEM. Journal of Vocational Behavior, 113, 51–61. https://doi.org/10.1016/j.jvb.2018.10.018

Supplementary Material

Please find the following supplemental material available below.

For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.

For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.

0.01 MB

0.00 MB