Competency Based Education Pilot in Illinois: Preliminary Findings

Abstract

In this study we employed regression analysis and between-group matched pair design to analyze whether participation in a competency-based education pilot was associated with improved high school completion and postsecondary entry. Data were obtained for high schools participating in a CBE pilot program in Illinois. Results of the matched pair design showed that CBE participation was significantly associated with higher rates of high school graduation for seniors across all groups with differences significant overall (97.2% for CBE vs. 93.1% for non-CBE), for White students (97.2% vs. 92.7%), and for students eligible for Free and Reduced Lunch (96.3% vs. 90.3%). Graduation rate for seniors was also higher for African American students and for Hispanic-Latinx students, but the differences were not statistically significant. In the matched pair design, CBE participation was not significantly associated with entry into college within 12 months of high school.

Keywords

education policy competency-based education mastery-based education proficiency-based education propensity score matching program evaluation

For this study, we evaluated the possible impact of the implementation of a competency-based education (CBE) pilot program on two overall student outcomes of interest, high school completion and postsecondary entry within 12 months of high school. This study was a part of a broader implementation and outcome evaluation of the competency-based education pilot we conducted on behalf of the Illinois State Board of Education (ISBE). We conducted an exploratory analysis of student data using a between-group propensity score matching and exact matching design. Our findings were mixed, with binary logistic regression results before the match showing a positive relationship between CBE participation and high school completion, but a negative relationship between CBE participation and entry into postsecondary (84 pilot high schools with 6,622 seniors in 2020, with 589 reported as participating in CBE). Further exploring the data through a matched pair design, we found that PSM results indicated that for the 481 matched pairs of students, CBE participation was significantly associated with improved high school completion but lower entry rates into postsecondary was no longer significantly related. However, the pilot had only been in place for 2 years so these results should be considered preliminary. We discuss this limitation in greater detail in the discussion.

Competency-based education is “a term that describes learning progressions based on mastery of content rather than passage of time… students understand learning objectives and also know what they must ‘know and show’ to be proficient” (Deye, 2017, para. 2). Versions of competency-based education have been around for decades but have gone by different names such as proficiency-based education, mastery-based education, and outcomes-based education (Evans et al., 2019, 2020; Guskey & Gates, 1986; Mitchell & Spady, 1978; Spady, 1977; Spady & Mitchell, 1977).

Early CBE-style reformers embraced Bloom’s (1968) model of learning for mastery. According to Bloom, “Our basic task is to determine what we mean by mastery of the subject and to search for the methods and materials which will enable the largest proportion of our students to attain such mastery” (p. 1). Advocates of CBE-style reform criticized traditional education as not meting the needs of many students and inhibiting their progress. Again, from Bloom, “… the problem of developing a strategy for mastery learning is one of determining how individual differences in learners can be related to the learning and teaching process” (p. 1). He cautioned against problems associated with traditional letter-grading (p. 2). Bloom noted several strategies for mastery learning to “find some way of dealing with individual differences in learners through some means of relating the instruction to the needs and characteristics of the learners” such as “a good tutor for each student” and more pragmatically, “Permitting students to go at their own pace, guiding students with respect to courses they should or should not take, and providing different tracks or streams for different groups of learners” (p. 7).

Over the last two decades, a new wave of CBE reform has grown in momentum. This wave builds on prior practices of mastery learning and student-centered practices. The four tents of this student-centered learning approach include: learning is personalized; learning is competency-based; learning happens anytime and anywhere; and students hold ownership over their learning (Reif et al., 2015). Personalized learning incorporates four strategies: learner profiles, personal learning paths, competency-based progressions, and flexible learning environments (Bill & Melinda Gates Foundation et al., 2014; Evans et al., 2020). In conjunction with CBE, this personalization is intended to help students to maximize their academic achievement because every student has different learning strengths, needs, and interests. CBE advocates have argued that K-12 education is employing outmoded instructional models that rely on grade-to-grade progression without demonstration of learning targets (Le et al., 2014). Working with teachers, students develop shared learning targets, and performance expectations, with progress marked by demonstration of proficiency rather than relying on seat time.

Clearly defining the parameters of CBE has been a challenge, but recent attempts have helped to clarify this. According to Evans et al. (2020), practitioners and policymakers working over two National Summits on K-12 Competency-Based Education in 2011 and 2017 agreed upon seven elements for a working definition of CBE:

Students are empowered daily to make important decisions about their learning experiences, how they will create and apply knowledge, and how they will demonstrate their learning.

Assessment is a meaningful, positive, and empowering learning experience for students that yields timely, relevant, and actionable evidence.

Students receive timely, differentiated support based on their individual learning needs.

Students progress based on evidence of mastery, not seat time.

Students learn actively using different pathways and varied pacing.

Strategies to ensure equity for all students are embedded in the culture, structure, and pedagogy of schools and education systems.

Rigorous, common expectations for learning (knowledge, skills, and dispositions) are explicit, transparent, measurable, and transferable.

State policy supporting and expanding CBE has been growing in the United States. According to the bipartisan National Conference of State Legislatures, a number of states have adopted varying degrees of competency-based education policies over the last few decades (Deye, 2018). According to a recent overview, 49 states plus the District of Columbia permit or facilitate CBE approaches (Aurora Institute, 2019). But this has not been without some controversy. For example, Maine passed a law in 2012 requiring students to show proficiency in eight content areas before earning a high school diploma with many schools moving away from traditional grading systems (Field & Feinberg, 2019; Miller, 2018). However, after resistance from many stakeholders who disagreed with the shift from traditional education, the requirement was repealed in 2018 in favor of allowing school districts to choose whether to employ CBE. Some Maine school districts have continued to favor the new proficiency-based system, while others have not.

The Illinois Context

In 2016 a pilot program was established under the Postsecondary and Workforce Readiness (PWR) Act (110 ILCS 148) (Illinois General Assembly, 2016) which authorized the implementation of competency-based education pilot programs in Illinois. The act enabled school districts to apply for a waiver to replace course-based high school graduation requirements with competency-based expectations. The program allowed schools to implement programs that changed from an education model based on traditional “seat time,” to a CBE model that allows students “to progress as they demonstrate mastery of concepts” (Illinois State Board of Education [ISBE], 2021, p. 1). As part of the pilot project, the ISBE provided a website with extensive supporting documents as well as made other training and support materials available to interested districts (ISBE, 2023). Twenty-six learning communities with over 100 schools/sites applied for the CBE waivers under the pilot program. These CBE programs were options for students with schools offering both CBE and traditional programs. The earliest adopters began implementation in 2018 to 2019. According to ISBE data, over 21,000 students had participated in CBE at the pilot sites in 2020, over 8,600 of those in high school.

In discussions with administrators at the CBE pilot sites in Illinois, we discovered that there were varying approaches to implementation of CBE. School administrators described differing styles and depth of pre-planning, training for faculty and staff, identification of program goals/outcomes, development of proficiencies/outcomes, etc. Some districts were heavily invested in implementation. Additionally, in some districts, there appeared to be considerable support for CBE implementation while in others, there was organized, and in some cases considerable, opposition to CBE. Extent of student participation varied as well. For some districts, the primary intent appeared to be to offer a new option to students who were having difficulty in the traditional classroom setting, whereas for others, the interest was more for students who wanted a different option that would allow them to set more challenging goals and pursue different paths toward achieving educational competencies. Historically, both concepts of CBE have been present in the literature. Earlier iterations of CBE had been associated with gifted programs, but more recently advocates have suggested it provides an opportunity for increasing equity for students who have had difficulty completing a traditional curricular approach. The pilot authorized districts to offer CBE coursework as an option for those students who wanted it. From our discussions with staff, in some cases students chose CBE options as an alternative because they needed more flexibility to successfully complete their degrees, while others were more interested in CBE as an option for adding new opportunities, flexibility and a more student-directed learning approach.

Competency Based Education and Student Outcomes

Evans et al. (2020, p. 3) conducted a systematic literature review of CBE finding that few studies have focused on the outcomes of CBE for K-12 students. According to the authors (p. 17), “The lack of research focused on student outcomes is likely due to the nascence of the reform and the need to better understand the factors that influence fidelity of implementation.” They add that “administrators have cautioned against conducting research on student outcomes for districts/schools in the early phases of CBE implementation because significant time is required to ensure CBE practices are implemented to their full potential and collecting data on student outcomes too early could produce misleading results (Pane et al., 2017; Scheopner Torres et al., 2015)” (p. 17). We agree with the authors that prematurely analyzing program outcomes is of concern, so we have been careful to describe our analysis as exploratory, not explanatory, despite employing a quasi-experimental design. We caution against drawing hasty conclusions from our analysis given the relatively recent implementation of the Illinois pilot. However, we believe that there is still value in analyzing the program to look for possible trends as it is being implemented.

Evans et al. (2020) found 12 studies examining outcomes of CBE for K-12 students. The authors categorized these as related to impact on (a) student academic achievement and progress; (b) student intrinsic motivation and engagement; and (c) other outcomes perceived as facilitators or barriers to student learning. They characterized nearly all of the studies as descriptive with limited generalizability, with just two at a higher correlational level of analysis. None were considered as experimental or quasi-experimental. Evans et al. considered the findings related to both “academic achievement and progress” and “intrinsic motivation and engagement” as mixed. For academic achievement, six studies showed some association between CBE and improved student achievement and progress, two did not. For intrinsic motivation, three studies showed some association between CBE participation and increased student intrinsic motivation and two showed no association. The four studies on “other student outcomes” they summarized this way: “In general, changes in attendance and student learning capacities were reported as positively impacted by CBE, but students’ perceived sentiment toward some CBE practices was sometimes negative (e.g., assessment policies, teaching, and grading practices)” (p. 19). The authors concluded that “Overall, these studies provide little definitive information about the relationship between CBE implementation and student outcomes” (p. 20). In our study we begin to address this gap in the literature by providing the first such quasi-experimental study to assess possible CBE impacts on high school completion and postsecondary entry.

Lastly, CBE advocates have argued that competency-based reforms could promote equity by shifting the focus away from traditional grading and seat time methods to promotion and completion based on demonstrated mastery of competencies and by emphasizing student agency in the educational process (Evans et al., 2019, 2020; Lewis et al., 2014; Lopez et al., 2017). However, as Evans et al. (2020) pointed out, little evaluation of equity outcomes has been provided in the literature. Equity considerations have been emphasized in the implementation of the Illinois pilot, and we have included them in our analysis by checking for differential impact across diverse racial/ethnic and income groups.

Design and Research Questions

For this exploratory study we first conducted binary logistic regressions, then a matched pair analysis using student record level data from the high schools participating in the CBE pilot in Illinois. The data were obtained for 84 Illinois high schools that chose to participate in a competency-based pilot program. We asked the following research questions:

(1) Controlling for several student academic, demographic, and environmental covariates, is participation in CBE associated with improved student high school completion by seniors?

(2) Controlling for several student academic, demographic, and environmental covariates, is participation in CBE associated with improved student postsecondary enrollment outcomes?

To answer the two questions, we examined anonymized student level data provided by the Illinois State Board of Education. We analyzed the data first via binary logistic regression analysis, then through a between-groups matched pair design. Matched pair design is a form of quasi-experimental design used to study the treatment effects of an intervention by comparing the outcomes for two different groups, those who participated in the treatment (in this case CBE participation) and those who did not (Blankenberger, 2020; Blankenberger et al., 2021; Gravezetter & Forzano, 2009). The Illinois pilot sites in the study offered students the choice of participating in both CBE and traditional programs making this an excellent opportunity to utilize a matched pair design. Matching allows a researcher to attempt to control for a variety of other factors that may impact likelihood of achieving an outcome. As the framework proposed by Perna and Thomas (2006) advised, student performance is affected by a variety of contextual factors. Numerous studies of educational performance have argued for the importance of considering context in studies of student educational outcomes (e.g., Attewell, 2001; Dumais, 2002; Espenshade et al., 2005), but we chose the Perna and Thomas framework. This structure is ideal for organizing studies with large datasets and multiple variables and has been used in many studies related to the impact of numerous factors on educational outcomes (e.g., Blankenberger, Lichtenberger, & Witt, 2017a; Blankenberger, Lichtenberger, Witt, & Franklin, 2017b; Gehlhausen Anderson & Blankenberger, 2023; Lichtenberger et al., 2014; R. Chen, 2012; R. Chen & DesJardins, 2010; LaSota & Zumeta, 2016; Lichtenberger & George-Jackson, 2012). Broadly reviewing the literature, Perna and Thomas developed a conceptual framework to help policymakers understand these factors and the potential impact they have on student outcomes. The authors argued that assessment of student success should be conceptualized within multiple contextual layers; internal student context, family context, school context, and the overall social, economic, and policy context. Each contextual layer includes multiple factors that can impact student success, and hence should be considered when evaluating educational policy outcomes. Accordingly, we have included factors from these multiple contextual layers as covariates in our design.

To determine whether a program/treatment yielded a desired outcome, in ideal circumstances, an evaluator would employ an experimental research design which randomly assigned participants to experimental and control groups to help to account for important factors which could impact participant outcomes. However, this is not possible in most evaluation settings because typically individuals cannot be randomly assigned to treatment and control groups. This may occur for a variety of reasons. For example, the treatment being evaluated may have already concluded, or it cannot be denied for ethical reasons, or there is no way to successfully insulate the experimental group from key factors which could impact the relationship between the independent and dependent variables (Gravezetter & Forzano, 2009; Rossi et al., 2003; Sylvia & Sylvia, 2012). This is typically the case in education program evaluations for a number of reasons. For example, students are not in controlled experimental situations, they cannot be randomly assigned, and they are affected by internal and external factors that may impact their performance (Blankenberger et al., 2021; Perna & Thomas, 2006).

When experimental design conditions are not possible, it is important to approximate the experimental condition by using quasi-experimental approaches which attempt to control for factors that may impact group outcomes (Austin, 2011; Blankenberger, Lichtenberger, & Witt, 2017a; Blankenberger et al., 2021, Author, 2017a, 2021). For our evaluation, we employed propensity score matching (PSM) and exact matching to create matched groups of students to attempt to mimic the randomization process (Austin, 2011; Author, 2021; Rosenbaum & Rubin, 1983; F. Thoemmes, 2012; F. J. Thoemmes & Kim, 2011). The intent of the matching process is to reduce bias by decreasing potential imbalance in pre-treatment confounders between the treated and control groups (Stuart, 2010), hence attempting to mimic the randomization process of experimental design. For our match, we employed the propensity score matching evaluation technique described in Blankenberger et al. (2021) to analyze the anonymized student level data from the CBE pilot sites for AY2020 provided by the ISBE. This dataset included a large number of student level factors which we could then employ as covariates for the match. Results of preliminary binary logistic regressions enabled us to identify several other factors from the dataset that may be associated with student outcomes, and we included these in the match process. Furthermore, we matched students within their high schools, so that as many high school covariates as possible would be controlled as well.

Propensity scores are the predicted probabilities of student participation in a treatment group. To obtain the probability score, we ran a logistic regression model with membership in the CBE group as the dependent variable and the identified covariates as the predictor variables (Austin, 2011). Depending on the evaluation, the matching process may be improved by using exact or coarsened exact matching on key factors that the researchers feel are critical to the analysis (Blankenberger et al., 2021; King & Nielsen, 2019). In our case, given the state’s equity concerns, we knew that examining differential impact by race/ethnicity and income level was critical to the evaluation, so we matched exactly on race/ethnicity and on Free and Reduced Lunch (FRL) eligibility as our proxy for family income level. The preliminary regression also indicated that English Language Learner status was associated with likelihood of participation in CBE, so we matched on it as well. To generate the PSM predicted probability score, we included SAT, and the number of college preparatory courses taken—honors, dual credit, Advanced Placement, and International Baccalaureate. We then matched the group members on this academic indicator to the nearest tenth (a caliper of 0.1, i.e., maximum difference in propensity score allowed within a matched pair) (Harris & Horst, 2016; Kim et al., 2016). Unfortunately, we were not able to use GPA because the State has not consistently collected this data. This is a limitation, as GPA is typically a strong predictor of both high school completion and postsecondary entry, the outcomes for our study (Allensworth & Clark, 2020; Blankenberger, Lichtenberger, Witt, & Franklin, 2017b; Blankenberger et al., 2021; Bowen et al., 2009; Geiser & Santelices, 2007). This likely impacted the sensitivity analysis results we report later in the article. In order to control for potential differences due to student and contextual factors, we matched students using both the propensity scores generated from the academic data (SAT, number of Honors courses taken, number of dual credit courses taken, and number of International Baccalaureate courses taken), as well as matching exactly on race/ethnicity, FRL, ELL factors, and on the high school attended. Since students in the study had the option to participate in CBE or not, this afforded us the opportunity to nest the comparisons within the high school, thus enabling us to control for high school context. That is, each matched pair of students attended the same high school limiting the impact of high school factors as possible confounders. Given the limitations in the data, this is especially valuable since we only have a yes/no value for participation in CBE, and we do not have data about the extent of CBE implementation at each school that we could use as control variables. Matching in high schools allows us to control for potential unknown confounders that accompany varying high school settings. Once the matched groups were created, we conducted chi-squares layered by race on our two outcomes of interest—high school completion for seniors and entry into postsecondary within 12 months of completion. Ideally, we would have tracked students from earlier in their academic careers, but we could not do so given the recency of the program’s implementation.

Although PSM is a widely used matching method (employed in tens of thousands of studies based on a Google scholar search use “propensity score” and “matching” or “match”), it is possible that PSM may increase imbalance between comparison groups and weaken inference by pruning too many observations during the matching process—what King and Nielsen (2019) refer to as the PSM paradox. We attempted to balance minimizing the loss of cases in the matching process, while still matching on key factors germane to the study for which we had data. We attempted to maximize the effectiveness of the matching process and address these concerns by employing a combination of PSM with coarsened and exact matching. Our intent in was to improve the balance in the comparison groups on key variables such as race/ethnicity and income level and others that we identified in preliminary regressions as important ones. If all factors are included in the PSM, this risks creating unbalanced pairs on some categorical factors. Knowing the importance of matching on race/ethnicity and income level to our analysis, we chose to match exactly on race/ethnicity and FRL, while including ratio measures in the PSM. We have provided a balancing table showing the differences in balance between the unmatched students and the matched students to show the differences in the groups before and after matching. Knowing that we could only control for the factors on which we have data (and notably we are missing GPA), we have also conducted a sensitivity analysis to determine whether the analysis may be sensitive to unknown confounders. That is, when there are too many missing data points to include in the PSM, the analysis may be sensitive to these potential unknown confounders. We feel we were able to match on as many students as we could while not sacrificing balance in the race/ethnicity and FRL factors. At the 2020 pilot sites, we had 5,767 seniors with the complete data necessary to conduct the match within their high school, and of that group, 554 seniors were listed as CBE participants at the pilot sites. At some high schools, only a few students were listed as CBE, at others, dozens were. We lost very few observations/students through the matching process, matching 461 pairs of students—all within their high school.

There were a few additional important design limitations to note. The earliest CBE pilot sites applied for participation in 2017, so their first implementation year was 2018 to 2019. Thus, even the first sites did not have a full 4-year high school cohort of students complete a CBE curriculum. Thus, the strength of the potential impact (either positive or negative) may not be fully evident. Furthermore, the nature of implementation varied substantially at the different sites. From discussions with ISBE and pilot site personnel, we discovered that some schools were implementing CBE in just a few courses, others in a few grades, and some were implementing it more broadly across the curriculum. However, the data reported to the ISBE by schools is a simple binary indicator with the student listed as either a CBE participant or not. The extent to which they participated and the manner in which the CBE courses were delivered is not indicated. Student may be taking one CBE course or an entire curriculum. There is no indication in this datapoint. In addition, more variability could be introduced due to the flexible and individualized nature of CBE coursework. Hence, this variability in delivery could impact the association with the student outcomes in this study. Given these limitations, we want to further emphasize that this analysis should be considered as exploratory. Additional longitudinal analysis should be conducted as data become available.

Results

Analysis With No Covariates

To answer the two research questions related to high school completion by seniors and entry into postsecondary, we started by comparing initial unmatched student outcome data to get a sense of what the descriptive data looked like. The first “unmatched pairs” reflect only non-experimental results incorporating no attempt made to control for student differences that could impact these outcomes. As Table 1 indicates, CBE students performed better on high school completion for seniors in high school, but worse on postsecondary entry within 12 months of high school. In both cases, the strength of association was very small. The latter negative results are mitigated in part in the matched pair analysis. We will return to the matched pair discussion below.

Table 1.

Unmatched Comparisons: CBE Versus Non-CBE Outcomes for Students at the Pilot Sites.

	N	HS seniors who graduated (%)	Transfers (excluded) (%)	Cramer’s V	Entered postsecondary within 12 months (%)	Cramer’s V
Unmatched students
Not CBE	6,024	88.5	3.2		55.7
CBE	589	94.6***	1.2	0.056	51.1*	0.026
White unmatched students
Not CBE	2,567	90.9	2.1		63.4
CBE	275	95.2*	0	0.052	59.7	0.023
African American students
Not CBE	830	79.4	6.2		38.9
CBE	114	91.5**	4.3	0.091	37.6	0.008
Hispanic-Latinx students
Not CBE	1,355	90.2	3.2		50.4
CBE	122	94.2	1.4	0.040	43.2	0.040
2+ races students
Not CBE	225	87.6	3.6		55.1
CBE	33	100	0	0.056	57.6	0.017

Indicates statistical significance at the .05 level, ** at .01, *** at .001.

Logistic Regression of Student-Level Data With Covariates

We conducted binary logistic regressions for the outcomes of high school graduation for seniors, and entry into postsecondary the following year to control for the effect of variables for which we had data (see Table 2). The first model was significant overall (Nagelkerke R ² = .334, p < .001, n = 6,419) indicating that the model accounted for 33.4% of the variation in high school completion. CBE participation was significantly associated with high school completion [B = 0.651, Exp(B) = 1.917, p < .05]. Exp(B) is the odds ratio in a logistic regression and indicates the predicted change in odds for a one unit increase in a predictor variable. An Exp(B) of greater than one indicates that the odds associated with the outcome, in this case high school graduation by seniors, were higher with a change in the predictor, participation in CBE coursework. The odds ratio signifies the strength of the relationship. In this case, while controlling for the other variables in the model, students who were reported as participating in CBE coursework had an odds ratio nearly a full point higher (1.917) compared to those not participating in CBE coursework. That is, holding all other variables in the equation constant, the odds of a senior identified as taking CBE coursework completing a degree were 1.917 higher than one not taking CBE coursework. We excluded the 203 transfer students from this regression since the data provided no indication as to whether the students graduated after transfer. The second model was also significant (Nagelkerke R ² = .308, p < .001, n = 6,622) indicating that the model accounted for 30.8% of the variation in entry into postsecondary in the next 12 months. However, there was a negative statistically significant relationship between CBE participation and enrollment in postsecondary [B = −0.287, Exp(B) = 0.705, p < .05]. That is, while controlling for all other variables in the equation, the odds of enrolling in postsecondary for a senior who was identified as taking CBE coursework actually decreased by 0.287 in the unmatched student analysis.

Table 2.

Bivariate Regression Results for CBE Participants at the Pilot Sites With Covariates and Senior High School Graduation and Postsecondary Entry Outcomes.

	Senior HS graduation (n = 6,419—transfers excluded)			% grades enrolled in postsecondary.w/in 12 months (n = 6,622)
	B	SE	Exp(B)	B	SE	Exp(B)
CBE participation	0.651	0.225	1.917***	−0.287	0.099	0.705*
District size (small-reference)
Medium	–0.669	0.574	0.512	−0.031	0.246	0.969
Large	–0.923	0.571	0.397	−0.199	0.246	0.820
% EBF capacity to meet expectations	–0.001	0.003	0.999	−0.001	0.002	0.999
School SAT reading	–0.008	0.008	0.992	0.002	0.004	1.002
School SAT math	0.003	0.006	1.003	0.001	0.003	1.001
% Enrollment white	–0.004	0.003	0.996	−0.003	0.002	0.997
% Enrollment low income	–0.010	0.004	0.990*	0.003	0.002	1.003
Race—(white—reference category)
African American	–0.087	0.133	0.917	−0.321	0.082	0.725***
Hispanic-Latinx	0.201	0.158	1.223	−0.192	0.079	0.825*
Asian American	–0.249	0.344	0.779	0.215	0.166	1.239
2+ races	–0.177	0.276	0.838	−0.275	0.150	0.760
Free reduced lunch	–0.482	0.116	0.618***	−0.724	0.064	0.485
IEP	–2.12	0.107	0.120***	−1.324	0.094	0.266
English lang learner	0.212	0.212	1.236	−0.445	0.138	0.641
Courses
# of AP	1.324	0.169	3.76***	0.596	0.031	1.814
# of dual credit	0.443	0.111	1.56***	0.393	0.040	1.481
# of IB	13.94	126,905	1,132,087	0.284	0.055	1.328
Nagelkerke R ² (change)	.334***			.308***

n = 5,767 students.

p ≤ .05.

***

p ≤ .001.

Matched Pair Design

We were able to match 461 CBE students with a non-CBE student within the same high school, and across multiple characteristics of interest for the matched pair analysis (see Table 3). There were 554 seniors listed as CBE participants out of a possible 5,767 seniors at the pilot sites for which we could conduct the match, so we only lost 93 students in the matching process. As Table 3 indicates, each student was matched with a student with the same race/ethnicity, Free and Reduced Lunch, and English language learner indicators and whose propensity score generated from the academic ratio measures was within one-tenth (i.e., a caliper of 0.1). Furthermore, students were only matched to someone within their high school to limit potential school related confounders. The balancing tables for the matched student groups indicate that the groups were balanced. The groups were compared on effect sizes for each ratio measured variable in the propensity score match, and by exact number for nominal variables. Although there is some disagreement in the literature on the cut off score for these balancing table differences, typically, a 0.2 effect size difference is considered acceptable, though 0.1 is preferable and more broadly accepted (Austin, 2011). Only the score for International Baccalaureate courses was above 0.1 for our matched groups. As described in greater detail in the methods section, the matched pair design tries to improve analyses by attempting to more closely approximate experimental conditions (Austin, 2011; Blankenberger, Lichtenberger, & Witt, 2017, Blankenberger et al., 2021). This type of quasi-experimental approach endeavors to lessen variable bias by reducing potential imbalance in pre-treatment confounders between the treated and control groups (Austin, 2011; Blankenberger et al., 2021; Rosenbaum & Rubin, 1983; Stuart, 2010), hence mimicking the randomization process of experimental design. The balance in matched groups indicates that we have been able to reduce these group disparities found in the unmatched students to within acceptable differences in effect sizes for the ratio measures, and by matching exactly on multiple key categorical variables. The largest differences reduced were for SAT, IB courses taken, race/ethnicity, FRL and ELL. We experimented with increasing our match caliper from 0.1 to 0.2 but this led to less balanced groups with effect size differences on some variables over 0.2, so we decided to stay with the lower caliper for the match (Harris & Horst, 2016; Kim et al., 2016). Here again, we were disappointed that the state does not collect GPA. Since we were matching within the high school, the potential variation in GPA by school would have been limited, and the importance of GPA as a predictor variable would have strengthened the analysis.

Table 3.

Balancing Tables for CBE Participation Comparisons.

	PSM matched			Unmatched students
	Mean or count		Difference effect size	Mean or count		Difference effect size
Variables	Control	CBE		Control	CBE
	N = 461	N = 461		N = 5,213	N = 554
SAT	951.91	957.77	−0.03	1,007.67	940.47	−0.34
# Honors courses	1.65	1.49	0.08	1.38	1.61	0.12
# AP courses	0.90	0.84	0.04	0.87	0.88	0.01
#dual credit courses	0.44	0.42	0.02	0.43	0.45	0.02
# IB courses	0.00	0.01	−0.13	0.09	0.0	−0.27
Free reduced lunch	216	216		2,254	261
ELL	6	6		244	17
Race
White	246	246		2,567	275
African American	88	88		830	114
Hispanic-Latinx	106	106		1,355	122
Asian American	5	5		242	6
2+ races	16	16		201	33

Of the 84 pilot high schools, we were able to create matched pairs in 55 high schools. As indicated in Table 3, we were able to match 246/275 (89.5%) of the White students, 88/114 (77.2%) of the African American students, 106/122 (86.9%) of the Hispanic/Latinx students and 5/6 Asian students, and 16/33 (48.5%) of the two or more race students. We were able to match 216 of 261 (82.8%) students participating in free and reduced lunch but only 6/17 (35.3%) English Language Learner students. Overall, we were very pleased with the high percentage of matches in nearly every group. Often this is not easy to do when employing exact matching in conjunction with PSM, especially when also matching within school. Given the importance of race/ethnicity and income in the analysis though, we wanted to be sure to match exactly by group. Since we were examining these groups both overall and individually, we would still be able to capture all differences even if some groups were a slightly different percentage in the matched pair group than in the original group.

Similar to the unmatched non-experimental data in Table 1, CBE participation was associated with higher rates of high school graduation for seniors. The differences were statistically significant overall, for White students, and for students eligible for Free and Reduced Lunch, although effect sizes were small in all three cases. Given the persistent racial/ethnic and income level student attainment gaps in Illinois and nationally (Blankenberger, Lichtenberger, Witt, & Franklin, 2017b), the consistency of completion across groups is encouraging. For context, in Illinois, the 4-year graduation rate (which differs from our indicator in that we only looked at seniors who graduated) was 88% for 2020 overall, 91.5% for white students, 79.9% for African American, 85.5% for Hispanic, and 82% for low-income students (ISBE, 2022). In the non-experimental analysis, CBE participation was associated with lower rates of entry into postsecondary. However, in the matched pair design, these differences were somewhat mitigated as the lower rates of completion were no longer statistically significant. Postsecondary entry was lower overall, for African American, for Hispanic/Latinx, and for Free and Reduced Lunch eligible students but the differences were not statistically significant. For context, the postsecondary entry rate for Illinois was 70% in 2020 (ISBE, 2022). Nonetheless, this difference is a cause for concern and merits monitoring as the program matures, especially given the emphasis on potential equity benefits suggested in the CBE literature.

We checked the robustness to hidden bias for the significant findings. As noted in the methods, this is important because matched pair design can only account for the factors included in the analysis, and it does not match on those not included. We conducted a sensitivity analysis using the Rosenbaum method (Rosenbaum, 2007) by using updated versions of the spreadsheets created by Cabral and Luiz (2007). In the three significant findings, the gamma (Γ) scores were close to 1 (see Table 3) suggesting that the conclusions about association may be sensitive to unobserved covariates (Cabral & Luiz, 2007; M. Y. Chen et al., 2020; Rosenbaum, 1995, 2007). This is to be expected since the known covariates included in the regression models only accounted for about one-third of the variation in the dependent variables. Hence, we have more unknowns than knowns. We attempted to control for as many factors as possible available to us in the dataset including matching in the high school itself, as well as by race/ethnicity, FRL, ELL, and several academic characteristics. Not being able to add high school GPA is particularly unfortunate as it is often strongly associated with both high school and postsecondary academic outcomes, but the state does not collect this data as it considers it too inconsistent from school to school. This is a limitation, as GPA is typically a very strong predictor of both high school completion and postsecondary outcomes (Allensworth & Clark, 2020; Blankenberger, Lichtenberger, Witt, & Franklin, 2017b; Blankenberger et al., 2021; Bowen et al., 2009; Geiser & Santelices, 2007). This limitation often occurs in real-world quasi-experimental design evaluations as researchers typically are analyzing available data (which is often limited) after the fact, as opposed to being able to employ a randomized controlled trial design. It is important to report results of sensitivity analysis for any such design used for program evaluation. This sensitivity to potential unobserved factors further supports our rationale to consider this analysis as exploratory, not explanatory (Table 4).

Table 4.

Matched Group Comparisons: CBE Versus Non-CBE Outcomes for Students at the Pilot Sites.

	N	HS seniors who graduated (%)	Transfers (excluded) (%)	Cramer’s V	Γ	Entered postsecondary within 12 months	Cramer’s V
Matched students
Not CBE	461	93.1	2.6			57.9
CBE	461	97.2*	1.3	0.096	1.4	53.6	0.185
White matched students
No CBE	246	92.7	2.8			58.5
CBE	246	97.2*	1.6	0.102	1.2	61.0	0.025
African American students
No CBE	88	90.9	2.3			48.9
CBE	88	97.7	1.1	0.147		38.6	0.103
Hispanic-Latinx students
No CBE	106	94.3	2.8			61.3
CBE	106	96.2	0.9	0.44		48.1	0.133
2+ races students
No CBE						75.0
CBE						56.3	0.197
Free reduced lunch
No CBE	216	90.3	4.6			45.8
CBE	216	96.31*	1.4	0.122	<1.2	37.5	0.009
Non-free reduced lunch
No CBE	245	95.5	0.8			68.6
CBE	245	98.0	1.2	0.098		67.8	0.085

Indicates statistically significant differences.

Lastly, we conducted binary logistic regressions using the matched student group data (see Table 5). The relationship between CBE participation and high school completion was positive and statistically significant [B = 0.961, Exp(B) = 2.614, p < .05]. That is, while controlling for all other variables in the equation, the odds of a senior completing high school who was taking CBE coursework was 2.614 higher than those not participating in CBE. This score was even greater than the odds differences for the regression results for the unmatched pairs. CBE participation was significantly associated with senior high school graduation but not enrollment in postsecondary. Hence, the positive association between CBE participation and high school graduation for seniors remained significant for both the regression results for the unmatched student model and for the matched student model. The significant negative association between CBE participation and postsecondary in the unmatched student model did not remain for the matched student model, and neither did the significant results for African American and Hispanic/Latinx students, though not surprisingly there was a significant negative association with FRL.

Table 5.

Regression Results for Matched CBE Participants With Covariates and Senior High School Graduation and Postsecondary Entry Outcomes.

	Senior HS graduation (N = 904, transfers excluded)			Grads enrolled in postsecondary.w/in 12 months (N = 922)
	B	SE	Exp(B)	B	SE	Exp(B)
CBE participation	0.961	0.463	2.614*	–0.253	0.155	0.777
Race—White (reference)
African American	0.408	0.566	1.504	0.366	0.236	1.441
Hispanic-Latinx	0.189	0.583	1.208	0.361	0.209	1.435
Asian American				–1.373	0.850	0.253
2+ races				0.029	0.435	1.029
Free reduced lunch	–0.137	0.477	0.872	–0.772	0.177	0.462***
IEP	–0.949	0.472	0.387*	–0.644	0.284	0.525*
English lang learner	–0.439	1.191	0.645	0.271	0.657	1.311
Courses
# of Honors	0.511	0.354	1.667	0.133	0.070	1.142
# of AP	–0.644	0.459	0.525	0.269	0.116	1.308*
# of dual credit	0.145	0.366	1.156	0.521	0.118	1.684***
# of IB				–10.89	14,200	0.000
SAT score	0.006	0.002	1.006**	0.003	0.001	1.003***
Nagelkerke R ² (change)	.192***			.331***

p ≤ .05. **p ≤ .01. ***p ≤ .001.

Given the large number of factors in the models, we thought that there could be interaction terms that may be significantly associated with the two outcomes. However, we were also concerned about the potential for inflating errors by adding numerous potential interaction terms. We conducted several analyses adding interaction terms for each factor in the model as CBE*<term>. However, there were almost no interaction terms in any model that were significant, and the Nagelkerke R ² did not increase much in any model even with several added interaction terms. In none of the high school graduation outcome models with added CBE* interaction terms were any of these terms significant, although in a couple of the models, the addition of interaction terms caused the CBE variable to no longer be significant. In the postsecondary enrollment model, we did find that CBE*SAT was significant over several iterations of the model. Furthermore, in one of these models, the addition of interaction terms for CBE*race, CBE*SAT, and CBE*FRL caused the CBE participation variable to become positively significant [B = 2.162, p < .05], though CBE*SAT was the only other significant variable in that model [B = −0.002, p < .05]. But this only occurred in one model with that combination of added interaction terms. Given the preliminary nature of study, we felt it better to provide the result of the base regression and the majority of models—that CBE participation was not significantly associated with postsecondary entry for the matched students.

Conclusion and Discussion

Analyzing student level data, we examined the potential impact of CBE on student performance, specifically for seniors to complete high school, and for postsecondary entry within 12 months of high school at Illinois high schools participating in a CBE pilot program. We believe this is the first large scale study to employ a matched pair design using PSM or PSM with exact matching to analyze the impact of CBE participation on student outcomes. Our results were mixed. Unmatched binary regression analysis indicated a positive and significant association between CBE participation and high school completion for seniors while controlling for a number of key factors. In the matched pair design, participation in CBE was significantly associated with higher rates of high school completion for seniors compared to their matched peers. There were gains across race/ethnic groups, though only the Overall, White, and Free and Reduced Lunch eligible scores were statistically significant. Binary regression analysis with the matched students continued to show a positive significant relationship between CBE and high school completion.

However, CBE participation was not significantly associated with postsecondary entry either overall, or for any demographic groups. Furthermore, in matched pairs of African American, Hispanic, and Free and Reduced Lunch students, rates were considerably lower for the CBE students compared to the non-CBE students, though the differences were not statistically significant. The overall regression model results for unmatched students indicated a significant negative association between CBE participation and postsecondary entry, but that result did not persist in the regression model with the matched pair students. This suggests that some of the reduced likelihood of entry into postsecondary was accounted for when only comparing students with similar demographic and academic characteristics. However, some of the rationales for CBE are based on the idea of providing alternative paths to high school attainment for those students who have not been as successful in standard educational settings. Theoretically, improvement could be accomplished by changing the focus from traditional grading and seat time methods to promotion and completion based on mastery of competencies and by emphasizing student agency in the educational process (Evans et al., 2019, 2020; Lewis et al., 2014; Lopez et al., 2017). Hence, it is possible that those students at the pilot sites who were struggling to obtain a high school degree in a traditional setting were being encouraged to try the CBE path instead. Thus, those students may have been more focused on completing high school and less on starting postsecondary. Our data do not enable us to make a clear determination about this, though data we included on college preparatory course taking may have helped to address this. Thus, the fact that the matched pairs changed the negative result from being significant, to not being significant suggests this may play a part in the dynamic.

It is possible that differences between traditional and CBE transcripts could have impacted postsecondary entry as well. In our discussions with them, ISBE staff and administrators at the CBE sites indicated that CBE transcripts could be difficult for college admissions offices to interpret, and that it could be a hindrance to postsecondary entry. There is also a possibility that the recent impact of COVID could have carried over to these students and impacted these outcomes. However, the matched pair design was intended to help control for the potential impact of COVID and other environmental factors. That is, each matched student was paired with someone in the same high school. So, whatever remaining COVID conditions were present would have been present for all of those matched in the same high school. Unfortunately, it was not possible to control for the way that environmental conditions may impact individuals differently, so we were not able to control for that type of potential interaction effect. These limitations and possible alternative explanations underscore the need for additional research on CBE participation and the student experience in order to gain a more complete understanding of the role of CBE.

We note again that this study is exploratory and should be considered preliminary due to some data limitations, but more importantly because the program has only been an option for students at the pilot schools for a couple of years. The first group of participating schools only began implementing the pilot program in AY2019, and the analysis was for AY2020 data, so the programs have not had time to mature or for students to participate in CBE programs for more than a year or two.

Furthermore, we do not have any data as to the degree of CBE programing in which the students participated. The CBE participation data indicator reported by the high schools is a simple binary yes/no in the state’s data. A student may be taking one CBE course, a few, or an entire curriculum; we have no way of knowing based on this data set. Additionally, the flexible individualized approach inherent in CBE could add even more variability to the student experience of CBE. Although unavoidable given the nature of the data point, this variability could have impacted the results of our analysis. As discussed in the background section, our discussions with state officials and staff at CBE schools revealed that for some districts, the primary intent of adopting CBE was to offer a new option to students who were experiencing difficulty in the traditional classroom setting, while for others, the interest was primarily for students and families who were seeking more challenging goals and the chance to pursue different paths toward achieving educational competencies. The former reflects the newer view of CBE advocates while the latter is more akin to the earlier approaches to CBE-style education as a type of gifted program. Hence, for some students and adopting schools, the primary goal may have been to improve high school completion pathways, while for others, the aim may have been to offer advanced students new, more challenging pathways that would be more aligned to college entry. We believe that this variability offers an opportunity for future research. Instead of a binary indicator, researchers might find that an indicator of intensity or scale, might yield a different result. For example, instead of simply replying yes/no to a student’s CBE participation, schools could be asked to report the number of CBE courses taken for each student, or even survey students or instructors for more details about their level of participation. Similarly, more in-depth qualitative data could be gathered to add nuance to future studies on CBE impact.

Moreover, as discussed in the analysis section, the significant findings in the matched student model showed sensitivity to potential unobserved covariate confounders. Given these limitations, we recommend additional longitudinal analysis, particularly once a cohort has finished a full 4 years of high school as CBE participants. We also suggest further clarifying the data on level of CBE participation instead of relying on a binary yes/no indicator.

Although preliminary, our results are important for considerations of equity. Like all states, Illinois has faced persistent racial/ethnic and income level student attainment gaps (Blankenberger, Lichtenberger, Witt, & Franklin, 2017b). Illinois had an overall 4-year graduation rate of 88% in 2020, with substantial gaps between White (91.5%), African American (79.9%), Hispanic/Latinx (85.5%), and low-income students (82%) (ISBE, 2022). Advocates have argued that CBE reforms could promote equity and reduce these gaps by moving away from traditional grading and seat time practices to advancement based on mastery of competencies and by stressing student agency in education (Evans et al., 2019, 2020; Lewis et al., 2014; Lopez et al., 2017), though to this point, there has been little evaluation of equity outcomes in the literature (Evans et al., 2020). Our study found preliminary support that CBE participation was associated with higher completion for all groups of students, though in the matched pair design, differences were only significantly associated with higher rates of high school graduation for seniors overall (97.2% for CBE vs. 93.1% for non-CBE), for White students (97.2% vs. 92.7%), and for students eligible for Free and Reduced Lunch (96.3% vs. 90.3%). Graduation rates for seniors was also higher for African American students (97.7% vs. 90.9%) and for Hispanic-Latinx students (96.2% vs. 94.3%), though the differences were not statistically significant. However, although the differences were not statistically significant, postsecondary entry within 12 months was lower overall, for African American, for Hispanic/Latinx, and for Free and Reduced Lunch eligible students in the matched pair design. This may be a function of who was invited to participate in CBE though. One of the rationales for the Illinois pilot authorization was that CBE participation could encourage students who were not as successful in the traditional educational setting. So these students may already be less inclined to pursue postsecondary education. It would be interesting to follow these students to discover whether an increase in secondary success might lead them to go on to postsecondary. However, a possible barrier to postsecondary entry may be the need to change connections between educational levels. For example, admissions systems at postsecondary institutions may not be as prepared to process the records of CBE students, or to help them to navigate the placement testing processes. Higher education class structures may also be too traditional for students who have had success as CBE students. The links to higher education are cause for potential concern and merit monitoring as CBE programs continue.

Finally, we would like to add a note about program evaluations such as this one. For pilot programs such as this, it is critically important to analyze policy impacts. However, when conducting program evaluations in real-world settings, researchers are often limited in what they are able to accomplish. Typically, they are analyzing available, often limited data from programs in operation, and they are not able to employ the ideal of the experimental randomized controlled design. Researchers should nonetheless attempt to conduct as sophisticated a design as possible to control for covariates and ideally to attempt to mimic experimental design to the extent that they are able. But there are always limitations inherent in real-world evaluations, so it is important to discuss study limitations and to report the results of sensitivity analysis when using PSM or matched pair design.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Bob Blankenberger

Author Biographies

Bob Blankenberger is a professor in the School of Public Management and Policy at the University of Illinois Springfield. He is a former Deputy Director of Academic Affairs and Student Success at the Illinois Board of Higher Education. His research focuses on educational policy, assessment, and educational attainment.

Rob Kerr is the Executive Director of Innovation and Opportunity at the University of Illinois Springfield. He is a former Deputy Director for Entrepreneurship, Innovation, and Technology at the Illinois Department of Commerce and Economic Opportunity. His research interests are public administration, education policy, economic development, and tax incentive programs.

Ty Dooley is an associate professor and the Director of the School of Public Management and Policy at the University of Illinois Springfield. Dr. Dooley's research areas include social justice, critical race, social equity, and community development.

References

Allensworth

E. M.

Clark

(2020). High school GPAs and ACT scores as predictors of college completion: Examining assumptions about consistency across high schools. Educational Researcher, 49(3), 198–211.

Attewell

. (2001). The winner-take-all high school: Organizational adaptations to educational stratification. Sociology of Education, 267–295.

Aurora Institute. (2019). iNACOL releases updates to the snapshot of K-12 competency education state policy across the United States. https://aurora-institute.org/blog/inacol-releases-updates-to-the-snapshot-of-k-12-competency-education-state-policy-across-the-united-states/

Austin

P. C.

(2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46, 399–424.

Bill & Melinda Gates Foundation, Afton Partners, Eli & Edythe Broad Foundation, CEE Trust, Christensen Institute, Charter School Growth Fund, Silicon Schools. (2014). A working definition of personalized learning. https://assets.documentcloud.org/documents/1311874/personalized-learning-working-definitionfall2014.pdf

Blankenberger

. (2020). Higher education policy environment and accountability. In Slagle

Williams

(Eds.), Practicum in Public Administration (pp. 11–43). Birkdale Publishers.

Blankenberger

Gehlhausen Anderson

Lichtenberger

. (2021). Improving institutional evaluation methods: Comparing evaluations using PSM, Exact and Coarsened Exact Matching. Research in Higher Education, 62(8), 1248–1275.

Blankenberger

Lichtenberger

Witt

M. A

. (2017a). Dual credit participation, college selectivity, and enhanced degree attainment. Educational Researcher 46(5), 259–263.

Blankenberger

Lichtenberger

Witt

M. A.

Franklin

. (2017b). Diverse students, high school factors, and completion agenda goals: An analysis of the Illinois class of 2003. Education and Urban Society 49(5), 518–545.

10.

Bloom

B. S.

(1968). Learning for mastery. Instruction and curriculum. Regional education laboratory for the Carolinas and Virginia, topical papers and reprints, number 1. Evaluation comment, 1(2), n2.

11.

Bowen

W. G.

Chingos

M. M.

McPherson

M. S.

(2009). Crossing the finish line: Completing college at America’s public universities. Princeton University Press.

12.

Cabral

M. D.

Luiz

R. R.

(2007). Sensitivity analysis for unmeasured confounders using an electronic spreadsheet. Revista de Saúde Pública, 41, 446–452.

13.

Chen

M. Y.

Liu

Zumbo

B. D.

(2020). A propensity score method for investigating differential item functioning in performance assessment. Educational and Psychological Measurement, 80(3), 476–498.

14.

Chen

(2012). Institutional characteristics and college student dropout risks: A multilevel event history analysis. Research in Higher Education, 53, 487–505.

15.

Chen

DesJardins

S. L.

(2010). Investigating the impact of financial aid on student dropout risks: Racial and ethnic differences. The Journal of Higher Education, 81(2), 179–208.

16.

Deye

(2017). Overview of K-12 competency-based education. National Council of State Legislatures. https://www.ncsl.org/research/education/overview-of-k-12-competency-based-education.aspx

17.

Deye

(2018). A look at competency-based education in K-12 schools. National Council of State Legislatures. https://www.ncsl.org/research/education/a-look-at-competency-based-education-in-k-12-schools.aspx#:~:text=The%20NCSL%20Student%2DCentered%20Learning,and%20student%2Downed%20learning%20environments

18.

Dumais

S. A.

(2002). Cultural capital, gender, and school success: The role of habitus. Sociology of Education, 75(1), 44–68.

19.

Espenshade

T. J.

Hale

L. E.

Chung

C. Y.

(2005). The frog pond revisited: High school academic context, class rank, and elite college admission. Sociology of Education, 78, 269–293.

20.

Evans

C. M.

Graham

S. E.

Lefebvre

M. L.

(2019). Exploring K-12 competency-based education implementation in the Northeast States. NASSP Bulletin, 103(4), 300–329.

21.

Evans

C. M.

Landl

Thompson

(2020). Making sense of K-12 competency-based education: A systematic literature review of implementation and outcomes research from 2000 to 2019. The Journal of Competency-Based Education, 5(4), e01228.

22.

Field

Feinberg

(2019, April 19). Inside Maine’s disastrous rollout of proficiency-based learning. The Hechinger Report. https://hechingerreport.org/inside-maines-disastrous-roll-out-of-proficiency-based-learning/

23.

Geiser

Santelices

M. V.

(2007). Validity of high-school grades in predicting student success beyond the freshman year: High-school record vs. standardized tests as indicators of four-year college outcomes. Center for Studies in Higher Education.

24.

Gehlhausen Anderson

Blankenberger

. (2023). Validation and living learning communities: An evaluation case study. Journal of College Student Retention: Research, Theory & Practice, 25(1), 76–100

25.

Gravezetter

F. J.

Forzano

L. B.

(2009). Research methods for the behavioral sciences (3rd ed.). Wadsworth.

26.

Guskey

T. R.

Gates

S. L.

(1986). Synthesis of research on the effects of mastery learning in elementary and secondary classrooms. Educational Leadership: Journal of the Department of Supervision and Curriculum Development, N.E.A, 43(8), 73–80.

27.

Harris

Horst

S. J.

(2016). A brief guide to decisions at each step of the propensity score matching process. Practical Assessment, Research, and Evaluation, 21(1), 4. https://doi.org/10.7275/yq7r-4820

28.

Illinois General Assembly. (2016). Postsecondary and Workforce Readiness Act, 110 ILCS 148.

29.

Illinois State Board of Education. (2021). PWR act overview. https://www.isbe.net/pwr

30.

Illinois State Board of Education. (2022). Illinois report card, 2020-21. https://www.isbe.net/Pages/ReportCardArchive.aspx

31.

Illinois State Board of Education. (2023). Competency pilot. https://www.isbe.net/competency

32.

Kim

D. H.

Pieper

C. F.

Ahmed

Colón-Emeric

C. S.

(2016). Use and interpretation of propensity scores in aging research: A guide for clinical researchers. Journal of the American Geriatrics Society, 64(10), 2065–2073.

33.

King

Nielsen

(2019). Why propensity scores should not be used for matching. Political Analysis, 27(4), 435–454.

34.

LaSota

R. R.

Zumeta

(2016). What matters in increasing community college students’ upward transfer to the baccalaureate degree: Findings from the beginning postsecondary study 2003–2009. Research in Higher Education, 57, 152–189.

35.

Wolfe

R. E.

Steinberg

(2014). The past and the promise: Today’s competency education movement. Students at the center: Competency education research series. Jobs for the Future.

36.

Lewis

M. W.

Eden

Garber

Rudnick

Santibanez

Tsai

(2014). Equity in competency education: Realizing the potential, overcoming the obstacles. Jobs for the Future.

37.

Lichtenberger

George-Jackson

(2012). Predicting high school students’ interest in majoring in a STEM field: Insight into high school students’ postsecondary plans. Journal of Career and Technical Education, 28(1), 19–38.

38.

Lichtenberger

Witt

M. A.

Blankenberger

Franklin

. (2014). Dual credit/dual enrollment and data driven policy implementation. Journal of Community College Research and Practice, 38(11), 959–979.

39.

Lopez

Patrick

Sturgis

(2017). Quality and equity by design: Charting the course for the next phase of competency-based education. Competency Works and iNACOL.

40.

Miller

(2018, June 27). Legislators vote to ease Maine’s proficiency-based education mandate, allow more ‘local control’. Portland Press Herald. https://www.pressherald.com/2018/06/27/lawmakers-vote-to-eliminate-proficiency-based-education-mandate-in-maine/

41.

Mitchell

D. E.

Spady

W. G.

(1978). Organizational contexts for implementing outcome based education. Educational Researcher, 7(7), 9–17.

42.

Pane

J. F.

Steiner

E. D.

Baird

M. D.

Hamilton

L. S.

Pane

J. D.

(2017). Informing progress: Insights on personalized learning implementation and effects. RAND Corporation.

43.

Perna

L. W.

Thomas

S. L.

(2006). A framework for reducing the college success gap and promoting success for all. National Symposium on Postsecondary Student Success.

44.

Reif

Shultz

Ellis

(2015). A qualitative study of student-centered learning practices in New England high schools. Nellie Mae Education Foundation.

45.

Rosenbaum

P. R

. (2002). Sensitivity to hidden Bias. In: Observational Studies. Springer Series in Statistics. Springer, New York, NY. https://doi.org/10.1007/978-1-4757-3692-2_4

46.

Rosenbaum

P. R.

(2007). Sensitivity analysis for m-estimates, tests and confidence intervals in matched observational studies. Biometrics, 63, 456–464.

47.

Rosenbaum

P. R.

Rubin

D. B.

(1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.

48.

Rossi

P. H.

Lipsey

M. W.

Freeman

H. E.

(2003). Evaluation: A systematic approach. Sage.

49.

Scheopner Torres

Brett

Cox

. (2015). Competency-based learning: Definitions, policies, and implementation. Education Development Center Inc.

50.

Spady

W. G.

(1977). Competency based education: A bandwagon in search of a definition. Educational Researcher, 6(1), 9–14. https://doi.org/10.3102/0013189x006001009

51.

Spady

W. G.

Mitchell

D. E.

(1977). Competency based education: Organizational issues and implications. Educational Researcher, 6(2), 9–15. https://doi.org/10.3102/0013189X006002009

52.

Stuart

E. A

. (2010). Matching methods for causal inference: A review and a look forward. Statistical Science, 25(1), 1–21.

53.

Sylvia

R. D.

Sylvia

K. M.

(2012). Program planning and evaluation for the public manager (4th ed.). Waveland Press, Inc.

54.

Thoemmes

(2012). Propensity score matching in SPSS. http://arxiv.org/ftp/arxiv/papers/1201/1201.6385.pdf

55.

Thoemmes

F. J.

Kim

E. S.

(2011). A systematic review of propensity score methods in the social sciences. Multivariate Behavioral Research, 46, 90–118.