Abstract
We investigated environmental supports, an understudied aspect of Social Cognitive Career Theory (SCCT), with rural Appalachian youth, an understudied population. We developed a new measure of proximal influences that bolster the pursuit of postsecondary education, the Assessment of Postsecondary Supports (APSS). Across 3 studies, we develop and validate the new measure, finding strong internal consistency and construct validity. Moreover, scores on the APSS were positively correlated with college-going self-efficacy (CGSE) and college outcome expectations (COE), explaining more unique variance than a measure of perceived educational barriers.
Public Significance
Understanding proximal variables that help or hinder pursuit of education after high school is instrumentally important for bolstering participation in postsecondary opportunities and preparing for an increasingly educated workforce, especially students from under-privileged communities.
The landscape of career preparation has changed drastically over the last 50 years. In 1973, 28% of workers had some type of postsecondary education (Carnevale et al., 2013). Today, 48.4% of adults over age 25 have completed a two- or four-year degree (U.S. Census Bureau, 2022). Additionally, as the U.S. job market continues to demand more specific skills from the workforce, the proportion of jobs requiring postsecondary education is steadily increasing (Hoffman et al., 2011). As a result, prospective college students today face a holistically different environment than did previous generations. Understanding the mechanisms helping or hindering the pursuit of postsecondary education, then, has become of increasing importance.
Since rural Appalachia is an area where 70% of adults have no postsecondary education (Pollard et al., 2023), understanding educational supports and challenges in this region is especially important. Appalachia spans over 200,000 square miles of mostly mountainous land, stretching from Alabama to New York. Of the 25 million people living in the region, nearly half (42%) are considered rural (Appalachian Regional Commission [ARC], 2017). Although research on this population has been steadily increasing, more information is needed. Therefore, we sought to understand how supports for educational and career pursuits might be measured within the frame of Social Cognitive Career Theory (SCCT; Lent et al., 1994).
Rural Appalachian Culture
Though real challenges face rural Appalachia, the region unfortunately is often described only by its disparities and not by its unique strengths, which include awe-inducing vistas and a rich cultural heritage. Gore et al. (2011) described rural Appalachia as a sub-collective within the wider region, with community values indicative of a collectivist culture (see Triandis et al., 1988), in contrast to the individualistic culture prevalent elsewhere in the United States. (Gibbons et al., 2019) highlighted other strengths of this region, including important kin and extended family relationships, a strong sense of place and respect for the land, and appreciation for creativity and natural beauty.
Although the strengths of rural Appalachia are largely understudied, more is known about the alarming long-term health and educational disparities. For example, only 18.5% of rural Appalachian residents obtain a bachelor’s degree, compared with 31% in metropolitan communities across the region (Pollard et al., 2023). This partially explains why rural Appalachia has a disproportionately higher percentage of persons living beneath the poverty line (19.7%) compared to the wider Appalachian region (14.5%) and the nation (12.6%; Pollard et al., 2023). The mortality rate in rural Appalachia is significantly higher than the rest of the country, particularly for heart disease, diabetes, and cancer, suicide, and death from overdose (Meit et al., 2017).
Contributing to and exacerbating these disparities, changes to job opportunities also negatively impact rural Appalachians; the decrease in coal mining, logging, and manufacturing careers resulted in significant regional job losses (Hodge, 2016). Research also highlights career barriers for youth, including lack of college-educated role models, the need to prioritize family responsibilities over individual interests, and infrastructure challenges such as poor roads and lack of stable internet (Boynton et al., 2013; Pollard et al., 2023). Inequities in health and education in rural Appalachia are real. Reducing them, however, requires understanding how to build on strengths while also considering challenges. Such work is vital to supporting the career and educational development of rural Appalachian high school students.
Theoretical Basis
Over the last three decades, Social Cognitive Career Theory (SCCT; Lent et al., 1994) has been a prominent model for understanding choice behavior (Brown & Lent, 2016). The model suggests that core person and contextual variables such as self-efficacy, outcome expectations, goals, and proximal and distal influences, explain how individuals develop specific educational and work-related interests, create plans to achieve their goals, and perform within their chosen fields (Lent et al., 1994). As the core constructs in the model, the bulk of SCCT research has centered on the ways in which self-efficacy and outcome expectations explain interests, goals, and choice behaviors (Lent & Brown, 2006; Rottinghaus et al., 2003). Among rural Appalachian high school students, higher self-efficacy and outcome expectations scores were positively correlated with the aspiration to pursue a postsecondary education (Ali & McWhirter, 2006), in general, and STEM careers, in particular (Rosecrance et al., 2019).
Proximal Contextual Influences and the Importance of Supports
Increasingly, SCCT-based research has broadened its focus from the core constructs of self-efficacy, outcome expectations, and interest/goals/behaviors to include “contextual influences proximal to choice behavior” (see Lent et al., 2001). These proximal factors can either facilitate (support) or impede (be a barrier to) career development. Although both barriers and supports are key parts of SCCT, more attention, historically, has been paid to barriers (see Lent et al., 2000; Lent & Brown, 2006). This is in part due to interest in career barriers that emerged separate from SCCT (e.g., Swanson et al., 1996). However, as attention to the role of supports has increased, so, too, has evidence for their importance. A recent meta-analysis (Brown et al., 2018) found that, averaging across nine different educational and career outcomes from 246 different studies, support scales accounted for more of the variance (10%) in SCCT predictors than barrier scales (3%).
In the realm of career and education, environmental supports are diverse. Studies identify kin, parents, teachers, siblings, and friends as significant sources of support related to school engagement, vocational self-efficacy, and educational persistence (e.g., Ali et al., 2005; Kantamneni et al., 2018; Kenny et al., 2003). Whereas some studies have focused on sources of support, others have focused more explicitly on types of support. Lent et al. (2002) identified eight categories of support for career choices described in interviews with undergraduates at four-year and technical colleges. These included “social support or encouragement, …personal strengths, … direct experience with career-relevant tasks, role models/mentors, expected outcomes, … financial resources, goal setting, and self/career exploration activities” (p. 67).
As with all social cognitive constructs, supports are ideally conceptualized as domain-specific (Lent et al., 2000). In other words, supports for pursuing math- or science-focused careers may be different than supports for career development more broadly, which may be different still than supports for pursuing postsecondary education. Although there are a number of measures of career- and education-related supports measures, none specifically focus on pursuit of postsecondary education. For example, Lent et al. (2001) developed a measure of “perceived math/science specific choice …supports” (p. 477). Although it refers to supports for specific college majors and career paths, it is not designed to assess support for postsecondary education in general. As another example, The Contextual Support for Postsecondary Planning Scale (CSPPS; Ali et al., 2011) includes “six scales…designed to assess support for postsecondary career planning for high school students from six sources [e.g., family, peers, school personnel, and community],” (p. 128, emphasis added). Although the scale includes items related to postsecondary education (e.g., “This person is helping me apply for college or jobs”), none of the items refer solely to postsecondary educational planning. Most items, in fact, do not refer to postsecondary education at all. Other example items include This person talks to me about my future career plans and This person talks to me about his/her job. Given that many careers do not require education beyond high school, support for postsecondary career planning is a broader construct than support for postsecondary education.
Thus, for scholars interested in understanding students’ educational choices, measures focused specifically on supports for postsecondary education are needed. Currently, there are postsecondary education-specific measures of self-efficacy (Gibbons & Borders, 2010; Hardin et al., 2021), outcome expectations (Flores et al., 2008), and barriers (Gibbons, 2005; McWhirter, 1997), but no measures of postsecondary support, apart from an unpublished dissertation (Yi, 2009). However, the measure developed by Yi, although comprehensive, (a) is long (59 items, with three sources of support), limiting its use along with other measures, particularly in educational settings where time is limited; and (b) uses a range of response stems and different response options for most questions, increasing the likelihood of participant confusion and fatigue.
The need for a well-validated, domain-specific, concise measure of supports for postsecondary education remains. In addition, relying on barriers to understand proximal contextual influences may be especially problematic for individuals from rural Appalachia. Results from a mixed-methods study (Gibbons et al., 2020) found that rural Appalachian students reported relatively low subjective perceptions of postsecondary education barriers using a well-validated quantitative measure of barriers (McWhirter, 1997, adapted by Gibbons, 2005). Qualitative methods revealed factors underlying how students perceived barriers to postsecondary education, several of which incorporated the idea of supports. For example, students demonstrated a “bootstrapper mentality” that led them to acknowledge barriers might exist but quickly identify personal strengths and other supports that would allow them to overcome the barrier. Other research supports the idea of self-reliance, a component of the bootstrapper mentality, as a factor directly and indirectly impacting help-seeking (Keller & Owens, 2022). Second, given that the culture in rural Appalachia is characterized by collectivist values (Gore et al., 2011), interpersonal harmony (Beaver, 1989), and a deep consideration of family (Keefe, 2005), support from family, friends, and community likely are especially salient in Appalachia compared to other more individualistic or independent. Finally, barriers to pursuing postsecondary education affecting rural student populations (e.g., greater socioeconomic challenges in college and parents with lower educational expectations and involvement; Byun et al., 2012) have been well studied, but focusing on barriers alone perpetuates a deficit lens that is all too common when viewing Appalachians.
Purpose of this Study
Our goal was to create a brief, psychometrically sound measure of contextual factors that facilitate the pursuit of postsecondary education (as opposed to supports for general postsecondary career planning or for specific types of postsecondary educational majors), which we name the Assessment of Postsecondary Supports (APSS). By creating such a measure, we hope to both advance SCCT-based research on educational choice as well as take a strengths-based approach to research in rural Appalachia.
In Study 1, we describe the development and initial validation of the APSS. We wanted the items in our measure to cover the breadth of sources and types of support. Prior researchers have identified several overlapping domains of support. Early work on social support within a health context identified five types of support in two categories: action-facilitating support (informational and tangible) and nurturant support (emotional, social network, and esteem; see Cutrona & Suhr, 1992). Later work grounded in SCCT (Lent et al., 2001) identified four “conceptual clusters” of support: (a) social support and encouragement (cf. emotional and esteem support); (b) instrumental assistance (cf. tangible support); (c) access to role models or mentors (cf. social network support); and (d) financial resources (a specific type of tangible support). Ali et al. (2011) built on this work by highlighting specific sources of support, including peers, family, school, and community. As described more fully below, we generated items to cover many of these broad domains and sources, but did not have any a priori expectations about the underlying factor structure of the APSS, and therefore took an initial exploratory approach. In Study 2 we refined and shortened the measure, using 2 new samples to confirm the factor structure of the reduced APSS. In Study 3 we administered the final 16-item APSS to another new sample.
In addition to considering the factor structure of the APSS, we also explored evidence of convergent and predictive validity. Consistent with SCCT (Lent et al., 1994, 2001), we expected scores on our new measure to be positively related to college-going self-efficacy and outcome expectations and negatively related to college-going barriers. Based on past evidence that supports account for more variance in SCCT-related outcomes than barriers (Brown et al., 2018), we expected the same to be true with our new measure. Finally, based on past research that prospective first-generation college students (PFGCSs) are less likely than their continuing generation peers to enroll or persist in postsecondary education (Wilbur & Roscigno, 2016), we expected that they would have lower supports and higher barriers scores.
Study 1: Initial Scale Development
The research team included individuals not only with significant experience in career development research and intervention, but who also identified as rural Appalachian (second author, other team members), who had significant experience partnering with rural Appalachian communities (all authors, other team members), or who identified as coming from rural non-Appalachian communities or being a first-generation college student. Team members reviewed items from existing, related supports surveys to identify exemplar items covering key types of support that could be readily adapted for our specific purpose. Through this process, eight items were adapted from an unpublished dissertation (Yi, 2009) capturing supports in three categories: concrete (e.g., “I have someone I can talk to about how to get into college”), general (“I have someone who encourages me to go to college”), and peer supports (e.g., “I have friends who think college is important”). We adapted another seven items from Lent and colleagues’ (2001) Contextual Supports and Barriers scale to capture supports from family (e.g., “I have family members that encourage me to attend college”), peers (e.g., “I get encouragement from my friends about planning for college”), and “other adults” (e.g., “I have access to a ‘mentor’ who could offer me advice and encouragement about planning for college”), as well as instrumental supports of finances (e.g., “I have enough financial support from my family to pursue going to college”) and academics (e.g., “I could get helpful academic assistance if I needed it”). We also created 10 new items to ensure there were at least three items in each of the above categories and to create a new category of perceived community support (e.g., “I can name someone in my community who has gone to college”), a key variable for the population (Bermingham, 2016) also recognized by Ali et al. (2011) but absent in most other measures of perceived support. This process resulted in a final pool of 25 items, with content capturing nurturant (relational) supports from (1) others in general, (2) friends, (3) family, (4) other adults, and (5) the community, as well as informational and tangible supports (6) in general and from (7) general academic and (8) financial resources.
Participants
The study was conducted as part of a National Institutes of Health (NIH) grant-funded STEMM (science, technology, engineering, math, and medical sciences) vocational and educational outreach program working with students from several high schools in rural Appalachia (see Gibbons et al., 2020, for a description of the program). Participants were 10th grade students attending three public high schools in two rural Appalachian counties participating in the grant-funded intervention program. Both counties were designated as economically distressed at the time of data collection, meaning they ranked in the bottom 10% of the nation on unemployment rate, per capita income, and poverty rate (ARC, 2017). Prior to the program intervention, students completed a battery of measures to establish baseline levels of key SCCT-related variables. To identify students who did not provide sufficient effort in taking the surveys, instructed response items (Meade & Craig, 2012) were embedded within each measure asking students to choose an answer (e.g., “Please select D”). Students who did not correctly answer the item for a particular measure were excluded. Additionally, only data from students who assented to have their responses used for research purposes and whose caretakers did not decline consent for research participation are included.
After screening for insufficient effort, we had 319 participants. Nearly half (152) of the participants were male, 160 were female, and six preferred not to answer. Reflecting the demographics of the area, our sample was largely white (93%).
According to Pollard et al. (2023), Appalachian TN is 86.4% white, Non-Hispanic, 5.5% Black or African American, and 4.7% Hispanic or Latino. Seventy-eight (25%) students were PFGCSs, defined as students who report neither adult caregiver pursued any education beyond high school. By this definition, 173 (55%) students were continuing generation students and 61 (20%) students reported being unsure of their caregivers’ education level.
Instrumentation
Initial Assessment of Postsecondary Supports
For each item on the initial 25-item APSS, students rate their level of agreement on a 5-point Likert-type scale from 1 (strongly disagree) to 5 (strongly agree). Higher scores reflect higher levels of perceived supports to pursuing a postsecondary education.
College Outcomes Expectations Scale
The 19-item College Outcomes Expectations Scale (COE; Flores et al., 2008) assesses the extent to which students perceive the pursuit of a postsecondary degree as beneficial (e.g., If I get a college education, then I will do well in life). Students rate their agreement on a 1–10 Likert-type scale, where higher ratings indicate greater agreement. Item responses are averaged, with higher scores representing more positive perceptions of attending college. The COE was created for use with high school students and has shown strong psychometric properties (Flores et al., 2008). COE scores were not affected by demographic variables, supporting divergent validity. Internal consistency in the current sample was high (⍺ = .94).
College-Going Self-Efficacy Scale
The College-Going Self-Efficacy scale (CGSES; Gibbons & Borders, 2010) measures the extent to which students perceive that they can get to and through college. Items include “I can choose a good college” and “I could fit in at college,” all answered with a 4-point Likert-type scale, 1 representing not at all sure and 4 very sure. Higher scores reflect higher college-going self-efficacy. The measure has been used with a variety of students in both middle and high school (Gonzalez et al., 2013). The scale showed strong internal reliability in an earlier study (⍺ = .94; Gibbons & Borders, 2010), as well as in the current study (⍺ = .95).
My Perception of Barriers
We used a 36-item version of the My Perception of Barriers scale (PEB; McWhirter, 1997; revised by Gibbons, 2005), which assesses the type and magnitude of perceived barriers to education and career pursuits. Gibbons augmented the original 28-item scale to capture salient themes not present in the prior version, yielding a 45-item scale. In the current study, we dropped several items that demonstrated poor utility in past research with our population (e.g., questions about racial discrimination) and added 2 items related to being treated differently for coming from a rural community and needing to take on adult responsibilities to yield a 36-item scale. This modification of the PEB-R is consistent with other published research. For example, in a 2018 study (McWhirter et al., 2018), McWhirter (author of the original PEB) and colleagues removed, altered, and added some items to better capture for their participants and research questions, resulting in a 24-item scale. Participants are asked to evaluate each item for the likelihood it would be a barrier to continuing education after high school. Responses are made on a 4-point Likert-type scale ranging from 1 (not at all likely) to 4 (definitely) with higher scores indicating more perceived barriers. The PEB has indicated good internal consistency in prior research (⍺ = .93; Gibbons et al., 2020), along with the current study (⍺ = .93).
Results and Discussion
Due to low item-level missing data (<5% on all items) available item analysis was used (see Parent, 2013). Seven outliers were identified; although the pattern of results did not change, relationships among variables were stronger with the outliers included. For our final analysis we used the more conservative model omitting the outliers, resulting in a final sample size of 312.
Determining the Factor Structure of the APSS
We submitted the 25 APSS items to a Principal Axis Factoring EFA using an oblique, Promax rotation: Kaiser-Myer-Olkin (KMO) = .928, Bartlett’s test = 4505.20 (df = 300, p < .001). Four factors had eigenvalues greater than 1; however, there was a significant drop after the first component, which accounted for 40.28% of the total variance. Using Cattell’s Scree test (1966), visual inspection confirmed a clear break after the first factor, followed by two additional breaks after the second and third, which accounted for 8.34% and 6.23% of variance, respectively.
Items and Loadings for One-, Two-, and Three-Factor Solutions for APSS, Study 1.
Bold items indicate high factor loadings (>.32), italicized items indicate high cross-loadings (<.15).
To further explore dimensionality (Courtney & Gordon, 2013), we used Horn’s parallel analysis and Velicer’s (1976) minimum average partial (MAP) tests. Using O’Connor’s (2000) syntax for generating the analysis in SPSS, we computed a model with 1000 sample comparisons. Factors were retained if the eigenvalue in the real data exceeded that of the mean eigenvalue from the randomly generated data (Courtney & Gordon, 2013). Results showed the raw data eigenvalues for the first factor (4.84) and second (1.40) were greater than the means of the generated data set (1.26 and 1.18, respectively), thus suggesting a two-factor solution. Results from Velicer’s revised MAP test also indicated a two-factor solution. Three items (2, 16, and 25) had low communalities (<.40); however, these items had high factor loadings (>.32; Worthington & Whittaker, 2006), with no significant cross-loading, and appeared to capture a distinct aspect of support that is known to be critical to pursuit of postsecondary education: finances. There were an additional two items (3 and 4) that had good communalities but did cross-load on both factors; however, because both items tapped important sources or types of support not captured by the other items and did exhibit strong loadings, we chose to retain them for subsequent analyses.
The first 10 items of the survey pertain to supports from friends and family (APSS-FF), while the last 15 items reflect school and community (APSS-SC) supports. We used means of the items to form these two subscale scores, which had strong internal consistency: APSS-FF α = .82 and APSS-SC α = .90. Although a two-factor structure was evident, the strong correlation between the factors supports the possible use of a total score, which also demonstrated strong internal consistency α = .94.
Construct Validity of the APSS
Variable Correlations and Descriptive Statistics From Study 1 (N = 312).
Notes: *p < .05; Cronbach’s alphas are presented on the diagonal.
CGSE = College-Going Self-Efficacy Scale; COE = College Outcome Expectations; APSS Total = Total scores on the Assessment of postsecondary Supports; APSS_FF = Friends and Family Subscale of the APSS; APSS_SC = School and Community Subscale of the APSS; PEB-R = Perceived Educational Barriers Scale, Revised; PFGCS = prospective first-generation college students; non-PFCGS = non-prospective first-generation college students; Unsure = students unsure of their parents’ educational attainment.
Partial Correlations and Unique Variance Accounted for by Postsecondary Barriers and Supports in Studies 1 and 3.
**p < .01; ***p < .001.
Note. In Study 1, the 25-item APSS was used; In Study 3, the final 16-item APSS was used.
These results provide preliminary support for the 2-factor structure underlying the APSS, as well as evidence that the APSS correlates with other theoretically relevant constructs in predicted ways. Moreover, the results suggest that, as expected, postsecondary supports account for significantly more unique variance in college-going self-efficacy and outcome expectations than barriers, indicating that the APSS may be a useful tool in applying SCCT to understand the educational aspirations and attainment of rural Appalachian youth. However, without cross-validating the 2-factor structure and replicating these results, conclusions about the psychometric properties of the APSS are limited. We also wanted to determine whether the underlying structure was invariant across PFGCS-status and gender, given prior research on differences in postsecondary enrollment and persistence for these groups, as well as differences between these groups on other important social cognitive variables, with PFGCSs (Wilbur & Roscigno, 2016) and men (Pew Research Center, 2023) typically showing lower enrollment/persistence in college, as well as lower college-going self-efficacy (Rosecrance et al., 2019). In addition, to facilitate use in educational settings, we wanted to ensure that the APSS included the fewest items needed for a valid and reliable measure of supports. Finally, we wanted to refine the items for consistency (i.e., making sure each item started with “I”). We therefore collected data from an independent sample of high school students to further refine and validate the APSS.
Study 2
Method
Data for Study 2 were also collected from the same rural Appalachian high schools with which we partner for our grant-funded research intervention project. The original grant (through which the Study 1 data were collected) ended in Spring 2020, and a new five-year grant that expanded the in-school career intervention curriculum to 9th graders began in Fall 2020.
Participants
We combined data from Fall 2019, Fall 2020, Spring 2021, and Fall 2021. Data were collected from 9th and 10th graders at four different high schools. We eliminated duplicate participants (i.e., those taking the APSS in both 9th and 10th grades) when they could be identified by their program-assigned code number. After also excluding students who did not provide assent (n = 74) or who did not correctly answer the instructed response item embedded in the APSS (n = 66), we had a final useable sample of 527. All participants completed the measures in-school via web-based survey.
Approximately half (53.7%) identified as female, 42.5% as male, and the remaining 3.8% identified as non-binary or declined to answer. Most of the samples (57.9%) were in the 9th grade, 41.4% were in the 10th grade, and less than 1% did not indicate their grade. The majority (57.7%) identified as continuing generation students, 22.4% reported that they were PFGCSs, and 19.0% reported that they were unsure of their parents’ educational attainment. For the participants for whom we had race/ethnicity data (∼55% of the sample), the majority (89.8%, n = 254) identified as white.
Instrumentation
All participants completed the 25-item APSS developed in Study 1. We made slight modifications to the item wording so that all items began with “I” in order to enhance consistency without changing any item meaning. For example, Item 9 was changed from, “My family would be proud of me for planning to go to college,” to “I think people in my family would be proud if I planned for college.” Due to technical and researcher error, Item 23 was omitted from the Fall 2020 surveys (n = 50), and Item 11 was misworded (and thus excluded from) the Fall 2019, Fall 2020, and Spring 2021 surveys (n = 386).
Results and Discussion
Prior to the main analyses, we considered ways to shorten the measure to maximize efficiency and utility. Although 25 items may be considered brief, our own experience in school-based settings and research by others (e.g., Omrani et al., 2019) highlighted the importance of using the briefest scales possible both to reduce participant fatigue and resistance to completing what may be perceived as redundant items, as well as to reduce the usage of classroom time, and thus maintain good relationships with our school partners. We therefore carefully considered the results from Study 1 to help us meet our goals of (a) maintaining coverage of all 8 content domains across the 2 identified factors while (b) reducing overall scale length substantially. We examined the CFA results from Study 1 to identify the item in each of the 8 domains that had the lowest loading. These were items 3 (support from others in general), 4 and 7 (peer support), and 10 (family support) from the Friends and Family factor and Items 12 (support from other adults), 16 (financial supports), 18 (concrete academic supports), 20 (general concrete supports), and 25 (support from community) from the School and Community factor. Items 3, 4, and 12 also did not clearly load on a single factor in study 1, with loadings >.30 on both factors. In addition, item 7 had the lowest loading of the 10 items on the Friends and Family Factor and Items 16 and 25 had the lowest loadings on School and Community factor). This left six items on the Friends and Family Factor and 10 items on the School and Community Factor, with two items in each of the original content areas from Study 1.
To allow us to have an independent sample on which to cross-validate any suggested model modifications that might arise, we randomly split our total sample of 527 into two subsamples of 264 and 265. Using a Monte Carlo data simulation technique, Wolf et al. (2013) recommended a minimum sample size of 120 for a 2-factor CFA with 8 indicators per factor and power of .80. This is slightly lower than the ubiquitious heuristic of at least 10 participants per item, which in our case would mean a minimum sample size of 160. Thus, our sample sizes of 264 and 265 should be adequate for the initial CFAs.
Final Standardized Factor Solutions for the Reduced APSS (Studies 2 and 3).
Notes: The model in both studies specified error covariances between items 5 & 6 and Items 23 & 24.
Test of Multigroup Invariance
Tests of Multigroup Invariance in Study 2.
Next, we tested for metric or weak factorial invariance by constraining all factor loadings to be equal and fixing the latent means to zero in all groups. We used relative fit indices (as recommended by Little et al., 2007), comparing this constrained model to the baseline configural model. As shown in Table 5, the RMSEA point estimate and both the CFI and TLI all supported factorial invariance. Next, we conducted an even stronger test of invariance by constraining the item intercepts to be equivalent across groups (see Little et al., 2007), freeing the latent variances in the second (and third) groups. Again, relative fit indices supported this stronger factorial invariance. Finally, we tested for equivalence of latent means by constraining them to 0 in all groups. Here, chi-square difference tests (as recommended for tests of invariance of latent means) were significant, indicating that there was not invariance of latent means.
Other Reliability and Validity Information
Cronbach’s alpha reliability for the 16-item total score was good (a = .90); alphas for the 6-item Friends and Family subscale (a = .83) and for the 10-item School and Community subscale (a = .89) were adequate. Consistent with the previous tests showing variant latent means, a 2 (gender) by 3 (college generation status) ANOVA showed significant differences in APSS total scores for both gender F (1, 499) = 22.18, p < .001, pη2 = .04 and generation status F (2, 499) = 23.66, p < .001, pη2 = .09. There was no interaction between gender and generation status F (2, 499) = 0.65, p > .52. Female students (n = 282) reported higher APSS scores (M = 3.35) than male students (n = 223, M = 3.14). Post-hoc Tukey’s tests showed that continuing generation students (n = 297) had higher APSS scores (M = 3.38) than either prospective first-generation students (n = 117, M = 3.12) or those unsure of their parents’ educational attainment (n = 91, M = 3.05). The latter two groups did not differ from each other. Results were similar for the subscales, with female students having higher scores on both APSS subscales and continuing generation students having higher scores on both subscales than the other two groups, which did not differ from each other.
These results provide strong support for the 16-item APSS. The two-factor structure demonstrates measurement invariance across both gender and prospective college student status, and scores on the APSS differ on these variables in ways consistent with past research. However, the large amount of missing data on Item 11 in Study 2, along with the fact that the entire 25-item original measure was administered, limited our confidence in these results. Thus, we administered the final 16-item APSS to a new sample from the same population to provide additional support for the underlying structure and validity of the measure.
Study 3
Method
We collected useable data from 521 students in Spring 2022 and Fall 2023 from the same four partner schools in rural Appalachia. Participants were in 9th (n = 189), 10th (n = 129), or 11th (n = 208) grades. There were 245 young men (46.6% of the sample), 266 young women (50.6%), 11 non-binary students (2.1%); four students declined to indicate gender. The majority (89.7%; n = 472) self-identified as non-Hispanic and 85.2% (n = 448) identified as white. All students completed the 16-item APSS and demographic items on either paper packets or via an online survey on their school-issued computer. A subset of the larger sample (n = 433) also completed the short form of the College-Going Self-Efficacy Scale (Hardin et al., 2021) and the College Outcomes Expectations scale (Flores et al., 2008). A subset of these participants (n = 86) also completed the 36-item version of the PEB-R (McWhirter, 1997; revised by Gibbons, 2005) used in Study 1.
Results and Discussion
A confirmatory factor analysis in MPlus 8, using the latent standardization method of model identification (see Little et al., 2007) and allowing 2 pairs of error covariances, showed the two-factor model provided a good fit to the APSS data: RMSEA = .072 (90% CI [.065, .079]); CFI = .929; TLI = .916. All factor loadings and the two error covariances were significant, and all standardized loadings were at or above .53, with the exception of Item 3, which had a loading of .32 (See Table 4). As in Study 2, Cronbach’s alpha reliability for the 16-item total score was good (a = .90); alphas for the 6-item Friends and Family subscale (a = .83) and for the 10-item School and Community subscale (a = .88) were adequate.
Construct validity of the APSS
Scores on the APSS were positively correlated with college-going self-efficacy (rs = .658, .561, and .604, ps < .01, for the APSS-Tot, APSS-FF, and APSS-SC, respectively) and college outcome expectations (rs = .542, .558, and .457, ps < .01, for the PSS-Tot, PSS-FF, and PSS-SC, respectively). As expected, APSS scores were negatively correlated with perceived educational barriers (rs = −.435, −.210, and −.479, ps < .05, for the APSS-Tot, APSS-FF, and APSS-SC, respectively).
We again used partial correlations to understand the unique contribution of supports and barriers in explaining college-going self-efficacy and outcome expectations. As in Study 1, APSS scores accounted for 1.4 to more than 3 times as much unique variance in self-efficacy and 8.2 .6 to more than 13 times as much unique variance in outcome expectations as did barriers scores (see Table 3). Indeed, barriers explained a non-significant 2.2% of the unique variance in college outcome expectations, and only 9.7% of the unique variance in college-going self-efficacy, compared to 30.0% and 24.8% unique variance, respectively, accounted for by supports.
A one-way ANOVA demonstrated significant differences in APSS Total scores based on college generation status, F (2, 572) = 30.72, p < .001, pη2 = .10. Continuing generation students had significantly higher APSS total scores (M = 3.32, SD = .44, n = 344) than both PFCCS (M = 2.97, SD = .57, n = 118, p < .001) and those who were unsure of their parents’ educational attainment (M = 3.05, SD = .52, n = 113, p < .001), who did not differ from each other (p > .41). Differences were also found on the two subscale scores Fs (2, 572) > 22.01, ps < .001. Continuing generation students had significantly higher APSS-FF and APSS SC scores (Ms = 3.54, 3.19; SDs = .44, .53, respectively) than PFGCS (Ms = 3.25, 2.80; SDs = .66, .61, respectively, ps < .001) and those who were unsure of their parents’ educational attainment (Ms = 3.25, 2.93; SDs = .58, ps < .001); the latter two groups did not differ from each other on either subscale (ps > .18).
The results of Study 3 are consistent with our prior studies and add further support for the psychometric properties of the 16-item APSS. A confirmatory factor analysis supported the same 2-factor structure found in prior studies, and the total score and subscale scores demonstrated good inter-item reliabilities in this sample of rural Appalachian high school students. Supports again accounted for more unique variance in both college-going self-efficacy and outcome expectations than did barriers and, as expected, PFGCSs reported fewer supports for postsecondary education than did continuing generation students.
General Discussion
Results from these three studies suggest the 16-item Assessment of Postsecondary Supports (APSS) demonstrates good reliability and validity within the rural Appalachian high school student population. As expected, greater perceived postsecondary supports were correlated with greater college-going self-efficacy, as well as greater college outcome expectations. Consistent with SCCT (Lent et al., 1994), it appears the perception of supports, both relational and instrumental, bolster how students perceive their ability to persist through a postsecondary educational opportunity (self-efficacy), as well as their perceptions about the value that opportunity would afford them (outcome expectations). The current study is consistent with previous research suggesting that supports play an influential role in affecting choice behavior in vocation (Jenkins & Jeske, 2017), as well as STEMM educational fields (Flores et al., 2008; Lent et al., 2003).
Consistent with past research (Brown et al., 2018), supports appeared more impactful than barriers: Not only did we find that the APSS provided incremental validity by explaining variance above and beyond an existing measure of barriers, the APSS explained 2–3 times more unique variance in CGSE and more than 10 times more unique variance in COE than barriers. This suggests that perception of supports toward seeking postsecondary education is a better predictor of rural Appalachian students’ college-going self-efficacy and outcome expectations than their perception of barriers to such opportunities.
It is possible that the magnitude of the difference in predictive ability of supports versus barriers is indicative of the regional culture, where a reliance on family and relational supports to overcome barriers has existed for generations and is a key value of the community (Welch, 2011). Indeed, interviews with rural Appalachian students revealed that the perception of barriers may be mitigated by contextual factors, such as the region’s “bootstrapper mentality” and a focus on a positive future mindset (Gibbons et al., 2020). Thus, a more nuanced interpretation of barriers results, in conjunction with supports, may provide the most accurate picture of proximal influences on rural Appalachian students’ postsecondary pursuits. Finally, as expected, PFGCS and students unsure of their parents’ educational history reported significantly less perceived support and significantly more perceived barriers than their continuing generation peers.
Considering the factor structure of the APSS measure, factor analyses suggested a two-factor solution of the APSS, consisting of supports from (a) Family and Friends and (b) School and Community, and results support using the subscales as separate scores. Analysis using the full scale suggested the survey presents a psychometrically sound unitary measure of perceived general supports for postsecondary education, as well. Previous research assessing educational supports from a variety of sources found that a global measure of general support may provide the most explanatory estimate of how students perceive postsecondary support and how that impacts future planning and behavior (Ali et al., 2011). Researchers may use either the total or subscale scores, depending on the research/intervention questions being explored.
Overall, then, APSS seems to be a useful tool for measuring the construct of postsecondary supports, which is theoretically and practically important. Although both barriers and supports are considered proximal environmental influences in SCCT, they represent different constructs, requiring different measures. As such, the APSS offers a useful tool allowing researchers to assess the relative contribution of barriers and supports in postsecondary choice behavior through a SCCT perspective.
Limitations and Future Directions
The nature of our population—rural Appalachian high school students in Tennessee—is both a strength and limitation of this study. It is a strength because rural Appalachia is an understudied region that continues to face educational, economic, and health-related inequities, and a focus on supports (in addition to barriers) is especially important in this region. It is a limitation because more research is needed before concluding that the APSS demonstrates good reliability and validity with other populations. Given the prevalent disparaging stereotypes about the region, results may be especially vulnerable to the effects of social desirability. For instance, students may be inclined to inflate supports as a means of protecting their cultural identity.
Furthermore, an analysis of postsecondary supports, in particular, may have been affected by recent moves by Tennessee’s legislature to bolster postsecondary attendance. In addition to providing all public school students with two free administrations of the ACT, Tennessee became the first state to offer high school graduates two years of community or technical college tuition-free. As a result, perceived supports may be unusually high among students in the state. Similarly, although having students complete the APSS in their high school classrooms is consistent with how the APSS would be used in future research and application, the influence of taking the survey in a classroom setting could make supports more salient than they otherwise might be. Finally, students may be reticent to report a lack of support from family or friends. Future studies should seek to replicate our results with students from other states, as well as other regions.
Lastly, we experienced methodological challenges that can be addressed in future studies. We inadvertently left off an item in Study 2, and although the Results of Study 3 assuage many of the concerns about this omission, it is important for future research to continue to provide additional evidence of validity and reliability. Relatedly, the homogeneity of our population limited our tests of invariance; future studies can explore this aspect of validity of the APSS with more racially and ethnically diverse participants.
Implications for Future Research
This study demonstrates the potential for a new instrument, the APSS, for successfully assessing postsecondary supports in high school students. Although developed with a focus on rural Appalachian youth, the survey may be useful with other populations as well, particularly those with collectivist identities. Researchers could also further explore how the APSS works within the SCCT framework. For example, other studies could examine the use of the APSS in path analyses or in longitudinal studies that more directly test some of the directional hypotheses of SCCT.
More research also is needed on the relationship between supports and barriers for underserved students. Prior studies of rural Appalachian students highlight the challenges with exploring perceived barriers (Gibbons et al., 2020), but less is known about the relationship between supports and barriers. For example, do students see these on a continuum, with increased supports indicating reduced barriers? Or, can students simultaneously perceive both supports and barriers to college-going? In either case, this information would help expand our understanding of these important constructs.
Implications for Practice
Past research into the career and educational choices of rural Appalachians has emphasized a need for culturally attuned measures and interventions (Boynton et al., 2013; Gibbons et al., 2019). The postsecondary supports scale validated in this study was created with this expressed purpose. In addition, the scale was developed specifically in response to evidence that relying on barriers may have limitations in this population (Gibbons et al., 2020). Indeed, results from the current study suggest supports may be an especially powerful predictor of postsecondary self-efficacy and outcome expectations in the region, which in turn should predict motivation and choice. Moreover, perceived supports appear to explain substantially more about these pursuits than perceived barriers. This is consistent with the “bootstrapper” cultural values of the region, where a fierce resilience in the face of challenges is bolstered by strong interdependence among members in the community. Future interventions in the region targeting postsecondary education may benefit from emphasizing these strong cultural supports, rather than focusing on overcoming barriers.
Career educators can use APSS responses to highlight supports already identified by students while purposely increasing supports students endorsed less often. For example, if a group of students fails to recognize family support but strongly identifies peer support for college-going, schools may increase their family outreach programming while continuing to encourage a positive peer culture for postsecondary education. If community support is an emerging strength, career educators might intentionally build on this resource by inviting community members to offer emotional or instrumental support as students prepare for life after high school.
In this study, we described the development and initial validation of the Assessment of Postsecondary Supports (APSS), a new measure that can be used within the SCCT framework to specifically assess supports for the pursuit of postsecondary supports. Results from our studies indicate preliminary support for this measure as a way to explore proximal influences on career and educational development. Given the challenges of quantitatively measuring perceived barriers and the importance of domain-specific measures, a well-validated and useful measure of postsecondary supports is a valuable addition for researchers exploring social cognitive career variables. While additional studies are needed to bolster and extend this research, it appears that the APSS is an appropriate scale for research and practice purposes. Unlike other supports measures that focus on broader postsecondary career planning or pursuit of specific disciplines, the domain-specific focus of the APSS makes it particularly relevant to research and practice focused on educational attainment and reducing educational inequities.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the National Institute of General Medical Sciences (R25 GM129177, R25 GM137365).
