“STEMming” the Swell of Absenteeism in the Middle Years: Impacts of an Urban District Summer Robotics Program

Abstract

This article reports findings from a quasi-experimental study of the impact of a summer robotics program for urban middle-grade students. The study focuses on student engagement, measured by school attendance rate the year following the program. Program students, who were nearly all low-income minority students, were matched to comparison students who did not attend summer school. After establishing baseline equivalence in attendance between the groups, the study found a statistically and educationally significant program effect on school attendance the following year, suggesting that high-interest hands-on educational activities can help maintain student engagement in school.

Keywords

middle school attendance engagement

Introduction

Attendance can be considered the most fundamental behavioral indicator of student engagement with school. Although the concept of engagement has multiple dimensions (e.g., Fredricks, Blumenfeld, & Paris, 2004; Lawson & Lawson, 2013; Marks, 2000; Newmann, Wehlage, & Lamborn, 1992), the simple attendance measure should not be ignored. Student engagement in learning must begin with getting students to school in the first place. Research has shown that attendance is strongly linked to achievement (Chang & Romero, 2008; Connolly & Olson, 2012; Gottfried, 2010, 2011a, 2013). In addition, classroom peers suffer academically from the absence of other students when teachers must help those students catch up and cannot provide all the learning opportunities needed by other students (Gottfried, 2011b). Because attendance is such a strong predictor of course grades and high school graduation (Allensworth & Easton, 2007; Balfanz, Herzog, & Mac Iver, 2007; Finn, 1989; Lan & Lanthier, 2003; Neild & Balfanz, 2006a, 2006b; Roderick & Camburn, 1999), as well as success in college (e.g., Credé, Roch, & Kieszczynka, 2010), focusing on ways to improve school attendance rates is crucial for increasing levels of college and career readiness.

The fact that chronic absenteeism is such a powerful predictor of student academic outcomes and is highest in urban school districts with large percentages of low-income students (Balfanz & Byrnes, 2012) suggests that it is an essential research agenda issue for the field of urban education. Attendance problems are almost certainly linked to the key dimensions of urban education highlighted by Milner and Lomotey (2014): city size (densely populated urban neighborhoods—and particularly those characterized by high levels of violence and with significant transportation challenges), student characteristics (particularly low socio-economic status associated with family challenges related to school attendance), and limited school resources (for addressing attendance-related issues faced by students as well as for providing high-quality learning opportunities). Low student engagement with school at the secondary level is undoubtedly related to the unchallenging and unengaging curricular opportunities that characterize urban schools (Milner & Lomotey, 2014). The current research study contributes significantly to the field of urban education through its use of rigorous research methods to evaluate a particular educational strategy with the potential to increase urban student engagement in learning and ultimately improve student outcomes.

Review of Literature

Although many students begin missing too much school for the first time in ninth grade, poor attendance patterns often begin increasing in the middle grades and become worse in high school (e.g., Mac Iver & Messel, 2012). Recent studies find higher rates of chronic absenteeism (defined as missing more than 20 days of school) in Grade 8 than in earlier middle grades (Balfanz & Byrnes, 2012) as well as alarming rates of middle-grade absenteeism in urban districts such as New York City and Baltimore (e.g., Balfanz & Byrnes, 2013; Mac Iver, Plank, Durham, Farley-Ripple, & Balfanz, 2008). For example, whereas rates of chronic absence in the middle grades in Baltimore have declined from more than 40% of students in the early 1990s, one in six students in the middle grades still missed more than 20 days in 2012-2013 (Maryland State Department of Education, 2015). Drop-out prevention interventions that begin in ninth grade often cannot overcome the prior habits of poor attendance in the middle grades (e.g., Mac Iver, 2007, 2011).

Although poor attendance among urban students may stem from personal and family issues, such as health issues or the need for child care for younger siblings or for additional income from teenagers’ employment (Clement, Gwynne, & Younkin, 2001; Corville-Smith, Ryan, Adams, & Dalicandro, 1998; Fine, 1991; Wagstaff, Combs, & Jarvis, 2000), missing school during the secondary grades can often be traced to low levels of motivation. It is therefore critical to address the motivation issues that influence attendance and find ways to increase student motivation.

As Eccles (2008) has so aptly summarized the crux of the motivation issue, it often boils down to two main questions in students’ minds about what happens in school: “Can I do the task?” and “Do I want to do the task?” The research of Eccles and her colleagues has documented how middle school practices contribute to declines in student motivation. Grading practices that emphasize comparisons across students are often associated with lower course grades as students make the transition to middle school. Lower grades have a negative impact on motivation at a psychologically charged time in students’ lives. In addition to “less personal and positive teacher-student relationships” in their middle school classrooms (Eccles, 2008, p. 28), students’ experience of more negative assessments leads many to doubt their ability to “do the task.” This often influences them to exert less effort so they will not feel like failures even after they have tried. In addition, classroom practices often do not help motivate students to “want to do the task.” Many lesson plans do not address students’ needs to experience a task’s intrinsic value, attainment value, and utility value for the future (Eccles & Midgley, 1989). When students perceive more learner-centered practices in their instructional experiences (e.g., opportunities for autonomy, collaboration with peers, and shared decision making; complex, challenging, and relevant learning tasks), they also demonstrate higher levels of motivation (Meece, 2003).

This focus on motivation is related to recent discussions of non-cognitive factors affecting academic performance. Farrington and colleagues (2012) emphasized the importance of developing an academic mind-set to influence behaviors such as attendance and exerting effort in class and homework assignments. The key components of an academic mind-set are as follows: “1) I belong in this academic community; 2) My ability and competence grow with my effort; 3) I can succeed at this; and 4) This work has value for me” (Farrington et al., 2012, p. 28). The process of helping students to internalize these beliefs can occur not only in the core academic classroom but also in elective activities that build a sense of competence and value in academic pursuits. As Balfanz (2009) puts it, middle-grade students need

high engagement electives that provide avenues for short-term success and positively recognize asymmetrical skills levels . . . Experiences like . . . robotics and chess in which students with good engineering or logic abilities but limited formal mathematics skills can demonstrate strengths are essential. (p. 9)

A number of studies have begun to investigate the impacts of robotics instruction, but the research evidence remains rather thin. Although many articles describe robotics interventions or provide anecdotal evidence of student engagement (e.g., Beer, Chiel, & Drushel, 1999; Grubbs, 2013; Mauch, 2001), the focus of most published research thus far has been on either student learning or attitudes toward science and STEM (science, technology, engineering and math) careers. Benitti’s (2012) literature review of robotics programs in schools identified just 10 articles that included quantitative measurement of student learning. The reviewed studies reported some positive academic learning effects. Most of the studies did not involve a rigorous design involving randomization or closely matched comparison groups with baseline equivalence. Additional studies not reviewed by Benitti also yielded some positive results. Coxon’s (2012) randomized intervention study found a positive effect of early adolescents’ participation in a First LEGO League robotics competition on a measure of spatial ability, though the effect on males was not found for females. Welch and Huffman (2011) reported a positive effect of robotics programming on high school students’ attitudes toward science in non-equivalent comparison group study. Although the literature on robotics programming for secondary students alludes to increased student engagement, we found no studies that directly measured the impact of such programming on student engagement or attendance.

More generally, our literature review indicated that rigorous research on specific interventions to improve student attendance remains in the early stages. There is evidence that efforts to increase family engagement are associated with increases in student attendance at the elementary level (Sheldon & Epstein, 2002), and family interventions with high-risk middle school youth had a positive effect on attendance (Stormshak, Connell, & Dishion, 2009). Sinclair, Christenson, and Thurlow (2005) found significant effects in an experimental study of the Check and Connect intervention on the attendance of high school students with special needs. Marvul (2012) found positive effects in a randomized study of an intervention that coupled daily phone calls home with a moral issues class and club football and basketball teams. Another randomized study found high school attendance rates to be significantly higher for students at small schools of choice in New York City than for the control group (Bloom, Thompson, & Unterman, 2010). Staff at these schools attributed this to increased academic rigor and personal relationships (Bloom & Unterman, 2013). In general, however, there is relatively little rigorous research focused directly on interventions for increasing student attendance rates.

Focusing more specifically on out-of-school time programs, we found mixed results regarding program impacts on school attendance. Dynarski et al. (2004) found a significant positive effect on middle school attendance in their study of 21st Century Community Learning Centers using a matched comparison group. But Gottfredson, Cross, Wilson, Rorie, and Connell (2010) and Lauver (2002) found no statistically significant effect of after-school program participation on school attendance or on most other achievement-related outcomes in two separate randomized studies of middle school students. Other reviews (e.g., Lauer et al., 2006; McCombs et al., 2011) have focused more specifically on academic achievement outcomes rather than attendance. Cooper, Charlton, Valentine, and Muhlenbruck (2000) specifically advocated the need for summer program evaluations to expand beyond academic outcomes and argue that “special attention be paid to measures of . . . attendance and discipline problems during the following school year” (p. 102).

Given the salience of attendance as a predictor of student achievement outcomes noted earlier, additional research on effective means of increasing attendance for at-risk students is particularly important. This study seeks to contribute to this research agenda by assessing the effects of an out-of-school time intervention aimed at increasing student interest in STEM and engagement with school by giving students hands-on experience with robotics.

Background on the Intervention

The focus of this study is a robotics summer learning program, developed and conducted by an urban school district. In this district, 85% of students were eligible for free or reduced-price lunch and 92% were African American or Hispanic. The intervention was targeted at high-needs middle school students and funded by an Investing in Innovation (i3) development grant from the U.S. Department of Education. The primary goal of this 5-week summer program was to provide additional out of school time focused on science and mathematics instruction and robotics so that enrolled students could increase their mathematics achievement and develop interest in technology and STEM college majors and careers. In their theory of change, program developers expected increased engagement with school—as evidenced by higher attendance the following year—for students participating in the hands-on robotics activity. In turn, increased school engagement was expected to improve achievement.

All district students in Grades 5, 6, and 7 were eligible to enroll in the program. Although the program specifically targeted students who were low performers in mathematics on the previous year’s state assessment, the program was not oversubscribed and all students who applied were allowed to enroll. The robotics program was one of three enrichment programs offered during the district’s 5-week-long STEM summer school for the middle grades. Students in the robotics, arts, and sports programs shared a common mathematics and science instructional component at the same sites throughout the city. Robotics instruction was provided for 2 hr daily at each of the sites. Teams of students began building their robots the first week of the program, and worked to improve their functionality throughout the program. Students engaged in practice competitions at their sites as well as two city-wide robotics competitions for all sites.

Summer program teachers were recruited entirely from the district. The project provided a week of pre-program professional development to equip teachers to lead students in constructing a robot and participating in a robotics competition. An additional week of pre-program professional development was also expected to lead teachers to deliver high-quality math instruction. The sessions focused on development of fact fluency and automaticity, individualized skill practice, and use of formative assessment data to inform instruction.

Analysis of program implementation fidelity focused on four key program components: professional development for teachers in mathematics, professional development for teachers in robotics, summer program mathematics instruction, and summer program robotics instruction. Instruction in mathematics and instruction in robotics were implemented with fidelity. All sites succeeded in providing 80 min of math instruction daily during the program, with at least 5 min of daily math fact practice and weekly assessments to monitor student progress and guide instruction. Robotics instruction was provided for 2 hr daily at each of the sites, and all sites took students to attend the two robotics competitions. Robotics professional development was delivered with fidelity, but the percentage of mathematics teachers attending the full professional development (68%) did not meet the threshold for implementation fidelity set by the district (90%).

Students’ program attendance ranged from 1 to 24 days (4%-100%), with a mean of 66% (and median of 71%). Average program attendance rates were attenuated due to the inclusion of students who dropped out of the program after attending only a few days,¹ but other studies of urban district summer programs also note problems with attendance rates (e.g., McCombs et al., 2011). Summer program attendance is not compulsory, and families often make decisions during the course of a summer program that prevent students from attending regularly.

In this article, we focus primarily on the following research question:

Research Question 1: Did the 5-week STEM robotics summer learning program have a positive impact on the following year’s attendance rate of middle school students (compared with a matched sample of students who did not receive any of the district’s summer programs)?

Method

Research Design

Random assignment of students to the summer school program or control condition was not possible for this study. Program attendance could not be compelled. The program was not oversubscribed and non-compliance in attendance in a randomized study would have been a high probability. A quasi-experimental design was therefore specified, using a closely matched comparison group of students who did not receive the summer school intervention. An alternative design could have compared summer program students who selected the robotics enrichment with those who selected the arts and sports enrichments but received the same mathematics and science instruction. Because we had no theoretical reason to believe that robotics was inherently more engaging than arts and sports activities, we did not pre-specify this design. We also had no prior evidence that the groups of students in each of the summer enrichment programs would meet the threshold for baseline equivalence in prior attendance rates. But because the alternative design involves a useful additional comparison group—students who received the same academic instruction but an enrichment activity other than robotics—we also conducted post hoc analyses based on this alternative design and report them with the primary analyses below.

Data and sample

The district shared program records and student administrative data with the research team. A total of 193 students within the specified grade levels, from about 70 different district schools, were enrolled in the summer robotics program in eight different sites throughout the city. A total of 166 of these students had complete data on both the baseline and outcome measures.² Treatment students included in the analysis were 94% African American, 87% eligible for free or reduced-price lunch, 19% special education students, and 72% male.

Table 1 summarizes the significant demographic and behavioral differences between the students who attended the summer school robotics program and the full group of district students who did not attend summer school. These groups differed particularly in their attendance rates during the regular school year. Given the large differences between summer program students and others, it was important to identify a closely matched comparison group with the treatment students with baseline equivalence on the pre-test measures of attendance and mathematics achievement. To this end, we combined propensity score and Mahalanobis metric matching using a caliper matching technique studied and recommended by Rubin and Thomas (2000). The Mahalanobis matching prioritized close matching on the pre-test measures (attendance and mathematics achievement) over all the other matching variables to ensure the closest possible match on these most important covariates. Only potential control students with no missing data and who did not attend summer school were included in the matching analyses. In this two-step method, all control participants meeting these requirements who were within ±0.2 of the estimated propensity score of each treated participant were identified as potential matches. Then, Mahalanobis metric matching on prior attendance and prior mathematics score was used to make a final selection of up to three matches for each treated student (3 to 1 matching).

Table 1.

Differences in Group Means Between Robotics Summer Program Students and Non-Program Students Before Propensity Score Matching, by Prior Grade Level.

	Grade 5			Grade 6			Grade 7
	Program students (n = 57)	Non-program students (n = 4,527)	Standardized mean difference	Program students (n = 63)	Non-program students (n = 4,388)	Standardized mean difference	Program students (n = 46)	Non-program students (n = 4,406)	Standardized mean difference
Male	0.72 (0.45)	0.48 (0.50)	0.53	0.70 (0.46)	0.48 (0.50)	0.50	0.76 (0.43)	0.49 (0.50)	0.67
Free/Reduced Lunch (FRL)	0.86 (0.35)	0.89 (0.32)	−0.07	0.87 (0.34)	0.88 (0.32)	−0.01	0.80 (0.40)	0.86 (0.34)	−0.17
Minority	0.91 (0.29)	0.87 (0.34)	0.17	0.97 (0.18)	0.88 (0.33)	0.53	0.96 (0.21)	0.88 (0.33)	0.41
Special education	0.12 (0.33)	0.15 (0.36)	−0.10	0.22 (0.42)	0.15 (0.35)	0.16	0.20 (0.40)	0.14 (0.35)	0.17
Overage	0.11 (0.45)	0.21 (0.41)	−0.37	0.27 (0.45)	0.21 (0.41)	0.15	0.20 (0.40)	0.22 (0.41)	−0.08
Changed schools	0.02 (0.13)	0.07 (0.25)	−0.38	0.06 (0.25)	0.08 (0.27)	−0.08	0.04 (0.21)	0.07 (0.25)	−0.12
Suspended	0.07 (0.26)	0.08 (0.27)	0.03	0.10 (0.30)	0.12 (0.32)	−0.09	0.09 (0.29)	0.12 (0.33)	−0.15
Summer school prior year	0.46 (0.50)	0.23 (0.42)	0.46	0.27 (0.45)	0.14 (0.35)	0.33	0.52 (0.51)	0.12 (0.33)	0.84
Prior math z score	0.02 (1.13)	−0.03 (0.99)	0.02	0.35 (0.82)	0.38 (0.92)	0.01	−0.12 (1.01)	0.06 (1.0)	−0.11
Prior attendance	96.9 (4.39)	95.0 (5.38)	0.42	95.96 (5.6)	94.84 (5.74)	0.18	97.07 (2.56)	94.35 (6.72)	1.11
Prior year characteristics of students’ post-intervention school
Enrollment	232 (141)	220 (135)	0.07	271 (146)	221 (136)	0.38	300 (161)	226 (138)	0.52
%FRL	81.2 (13.8)	84.9 (12.5)	−0.30	82.3 (14.5)	85.0 (12.5)	−0.21	82.6 (13.1)	85.3 (11.9)	−0.29
Charter	0.19 (0.40)	0.19 (0.39)	0.00	0.13 (0.34)	0.18 (0.39)	−0.18	0.07 (0.25)	0.17 (0.38)	−0.45
Middle school	0.28 (0.45)	0.25 (0.43)	0.06	0.30 (0.46)	0.25 (0.43)	0.09	0.22 (0.42)	0.27 (0.44)	−0.15
Middle high	0.16 (0.37)	0.13 (0.34)	0.06	0.22 (0.42)	0.13 (0.33)	0.25	0.22 (0.42)	0.13 (0.34)	0.18
Average math z score	0.04 (0.51)	0.02 (0.49)	0.05	−0.05 (0.46)	0.03 (0.48)	−0.14	0.05 (0.46)	0.02 (0.49)	0.14

Note. Standard deviations in parentheses.

Within each prior grade level (fifth, sixth, and seventh), we selected a comparison group matched with the treatment sample as described above. The propensity scores were estimated using logistic regression with linear terms for each covariate: the two prognostic covariates (prior attendance and prior mathematics achievement score), eight student characteristics (dummy variables for ethnicity, gender, free or reduced lunch status, special education status, overage for grade, school transfers, prior year summer school attendance, and suspension), and five covariates measuring characteristics of the student’s post-summer program school (enrollment, percent of students receiving free or reduced-price lunch, two dummy variables representing the three different gradespan types—K8, middle grades only, and middle high school—a dummy variable indicating whether the school was a district school or a charter school, and the school’s average state mathematics assessment z score in Grades 6 to 8 for the prior year). We matched on characteristics of the student’s post-summer program school to address potential school effects on the outcome achievement score that was measured more than 7 months after the end of the intervention. Because the choice of post-summer program school occurred before the summer intervention, the distributions on these variables, measured in the year prior to the intervention, were not affected by the intervention. All the matching was performed using nearest-remaining-neighbor matching, beginning with the most difficult to match treated participant (the one with the highest propensity score) and proceeding to the participant with the lowest propensity score.

Table 2 summarizes the means and standard deviations on all matching variables for the treatment and matched comparison group, by grade level, together with the standardized mean difference between the groups. On average, the absolute value of the standardized mean difference was 0.08 for baseline attendance and 0.02 for math achievement. Overall, the absolute values of the standardized mean differences range from 0 to 0.3, with a median of 0.06. Three quarters (77%) of these absolute values are smaller than 0.09 (and 96% are smaller than 0.11), indicating a very closely matched comparison group with the treatment group on all matching variables.

Table 2.

Results for Differences in Group Means Between Robotics Summer Program Students and Comparison Students After Propensity Score Matching, by Prior Grade Level.

	Grade 5			Grade 6			Grade 7
	Program students (n = 57)	Non-program students (n = 169)	Standardized mean difference	Program students (n = 63)	Non-program students (n = 189)	Standardizedmean difference	Program students (n = 46)	Non-program students (n = 128)	Standardized mean difference
Male	0.72 (0.45)	0.73 (0.44)	−0.03	0.70 (0.46)	0.70 (0.46)	0.00	0.76 (0.43)	0.76 (0.43)	0.00
FRL	0.86 (0.35)	0.89 (0.32)	−0.08	0.87 (0.34)	0.89 (0.31)	−0.06	0.80 (0.40)	0.80 (0.40)	0.01
Minority	0.91 (0.29)	0.92 (0.27)	−0.04	0.97 (0.18)	0.97 (0.16)	−0.03	0.96 (0.21)	0.96 (0.19)	−0.04
Special education	0.12 (0.33)	0.09 (0.29)	0.09	0.22 (0.42)	0.18 (0.39)	0.10	0.20 (0.40)	0.20 (0.40)	−0.02
Overage	0.11 (0.45)	0.12 (0.33)	−0.06	0.27 (0.45)	0.25 (0.43)	0.05	0.20 (0.40)	0.21 (0.40)	−0.04
Changed schools	0.02 (0.13)	0.02 (0.13)	0.00	0.06 (0.25)	0.05 (0.21)	0.07	0.04 (0.21)	0.06 (0.24)	−0.09
Suspended	0.07 (0.26)	0.07 (0.26)	0.00	0.10 (0.30)	0.06 (0.24)	0.11	0.09 (0.29)	0.12 (0.32)	−0.10
Summer school prior year	0.46 (0.50)	0.47 (0.50)	−0.03	0.27 (0.45)	0.25 (0.43)	0.05	0.52 (0.51)	0.46 (0.50)	0.13
Prior math z score	0.02 (1.13)	0.04 (1.10)	−0.02	0.35 (0.82)	0.37 (0.80)	−0.02	−0.12 (1.01)	−0.12 (0.93)	0.00
Prior attendance	96.9 (4.39)	96.8 (4.1)	0.02	95.96 (5.6)	96.39 (4.77)	−0.08	97.07 (2.56)	97.50 (2.39)	−0.17
Prior year characteristics of students’ post-intervention school
Enrollment	232 (141)	245 (164)	−0.09	271 (146)	274 (161)	−0.02	300 (161)	310 (157.1)	−0.06
%FRL	81.2 (13.8)	81.6 (17.8)	−0.03	82.3 (14.5)	82.0 (16.1)	0.02	82.6 (13.1)	83.6 (14.9)	−0.06
Charter	0.19 (0.40)	0.19 (0.39)	0.03	0.13 (0.34)	0.17 (0.38)	−0.14	0.07 (0.25)	0.14 (0.35)	−0.30
Middle school	0.28 (0.45)	0.23 (0.42)	0.10	0.30 (0.46)	0.34 (0.48)	−0.09	0.22 (0.42)	0.31 (0.46)	−0.22
Middle high	0.16 (0.37)	0.12 (0.32)	0.11	0.22 (0.42)	0.19 (0.39)	0.07	0.22 (0.42)	0.24 (0.43)	−0.06
Average math z score	0.04 (0.51)	0.07 (0.53)	−0.06	−0.05 (0.46)	−0.03 (0.53)	−0.02	0.05 (0.46)	−0.06 (0.54)	0.21

Note. Standard deviations in parentheses.

Measures

The primary outcome variable for this study was yearly attendance rate, calculated for all students from district administrative records in the year following the summer program. The treatment variable indicates whether a student was in the robotics program (coded 1) or the matched comparison group with no summer school (coded 0). All covariates were pre-specified and included as grand mean centered in the final model, regardless of their statistical significance. Besides the matching variables listed above, covariates also included the baseline school-level attendance rate for the school attended by each student the year after the summer program. This was included to control for any school-level factors influencing the outcome variable, student attendance rate.

Statistical Analyses

The purpose of analysis was to estimate the effects of the summer program on individual students’ attendance in school during the school year following participation in the program. Because students were nested in different sites for the summer school intervention, we used a two-level model. The treatment students were nested in eight summer treatment sites, and control students were nested together in a ninth, no-treatment site. The hierarchical model addressed the fact that nesting of students violates the assumption of independence of observations. Summer site effects were not a focus of the study and the model assumed homogeneity of the treatment effects across sites. This follows the constant block effect model described by Dong and Maynard (2013). Because students were not randomly assigned to sites and we did not expect student outcomes to be equivalent across sites, we used a random intercept model, commonly used to allow “the mean of the level-1 units nested under a common level-2 unit to vary between level-2 units” (Hayes, 2006, p. 389). All other effects were fixed.

Level 1: Students within sites

Level 1 describes the relationship between students’ outcomes, student-level characteristics, and their treatment status. The Level 1 model is

Y_{i J} = β_{0 j} + β_{1 J} T_{i} + Σ β_{2 s} X_{s i j} + e_{i j},

where Y_ij is an outcome for student i in site j; T_i is 1 if the student is the treatment group and 0 otherwise; X_ij is a set of S student-level covariates (described above) for student i in site j, measured in the year prior to treatment exposure and centered on the grand mean in the sample; and e_ij is a random error term for student i from site j, assumed to be independently and identically distributed across students within sites (i.e., the “within-site” residual).

Level 2: Sites

β_{0 j} = γ_{00} + u_{0},

β_{1 J =} γ_{10},

β_{2 s} = γ_{2 S}

where γ₀₀ is the grand mean of the outcome variable (attendance) and γ₁₀ is the main effect of treatment.

The set of γ_2S regression coefficients represent the relationships between students’ outcomes and the covariates, with each coefficient assumed to be constant across sites, U_0j, j = 1, . . . , J are fixed effects associated with each site effect, and are constrained to have a mean of zero.

All available covariates described earlier were included in the final model, regardless of their statistical significance. The purpose for including the prognostic covariates—pre-intervention achievement and attendance variables—was to control for students’ prior achievement and prior attendance and to increase the precision of our impact estimates. The other covariates were intended to control for important student and school characteristics and to increase the precision of our impact estimates. To test for baseline equivalence between the treatment and control students on attendance, we estimated a hierarchical linear model in the form specified above in which students’ prior year’s attendance was predicted by treatment status (controlling for grade-level dummy variables).

Results

As shown in the first panel of Table 3, the intercept of the baseline equivalence model, the estimated mean prior attendance rate in the control group, is 96.8% of days enrolled. The prior attendance rate of the treatment group is estimated to be 96.5%. This difference of 0.3 percentage points is small and not significant, t(648) = −0.406, p = .684, Hedges’s g = −.07. This indicates that the groups were extremely closely matched on prior attendance rates.

Table 3.

Results From Hierarchical Linear Models Predicting Baseline Student-Level Attendance Rates (Year Prior to Intervention) for Full Sample and for Subsample of Low-Performing Students in Mathematics.

	Full sample (n = 652)		Low-performing students (n = 227)
	Coefficient	SE	Coefficient	SE
Intercept	96.84***	(0.65)	95.86***	(0.95)
Treatment	−0.31	(0.76)	−0.42	(1.23)
Sixth grade prior year	−0.58	(0.38)	−0.61	(0.88)
Seventh grade prior year	0.55	(0.42)	1.72	(0.91)

Note. Grade-level dummy variables were grand mean centered.

p < .05. **p < .01. ***p < .001.

Using the random intercept model detailed earlier, we estimated the impact of the Robotics Summer Program on students’ subsequent attendance in middle school during the 2012-2013 school year. As shown in the first panel of Table 4, the intercept (adjusted mean attendance rate in the matched control group) was 95.6% of the days enrolled. The adjusted mean attendance rate of the treatment students was 1.4 percentage points higher, 97.0% of the days enrolled. This impact was both statistically significant, t(631) = 3.52, p = .001, and large enough to be educationally meaningful, with a Hedges’s g effect size of .26. Another way of stating the impact is that treatment students attended about 2.5 days more of the 180-day school year on average.

Table 4.

Results From HLM Models Predicting Student-Level Attendance Rates in the Year Following Intervention for Full Sample and for Subsample of Low-Performing Students in Mathematics.

	Full sample (n = 652)		Low-performing students (n = 227)
	Coefficient	SE	Coefficient	SE
Intercept	95.61***	(0.20)	93.82***	(0.46)
Treatment	1.38**	(0.39)	2.58**	(0.90)
Male	0.13	(0.39)	0.18	(1.04)
FRL	−0.71	(0.54)	−0.87	(1.28)
Minority	−0.26	(0.83)	−0.78	(2.25)
Special education	−0.46	(0.48)	−0.53	(0.87)
Average for grade	−0.03	(0.45)	0.98	(0.86)
Changed schools	−0.15	(0.87)	−1.27	(1.89)
Suspended	−0.82	(0.65)	−2.01	(1.41)
Summer school prior year	0.92*	(0.36)	1.12	(0.85)
Prior attendance	0.65***	(0.05)	0.70***	(0.08)
Prior math z score	0.75**	(0.21)	0.52	(0.86)
Sixth grade prior year	−0.13	(0.42)	0.31	(1.31)
Seventh grade prior year	0.64	(0.46)	2.12	(1.14)
Prior year characteristics of students’ post-intervention school
Enrollment	0.00	(0.00)	−0.01	(0.00)
%FRL	0.01	(0.02)	0.03	(0.05)
Charter	0.05	(0.56)	−0.51	(1.38)
Middle school^a	0.93*	(0.46)	1.62	(1.02)
Middle high	1.45**	(0.55)	1.96	(1.20)
Average math z score	−0.43	(0.59)	0.40	(1.47)
Average attendance	0.10	(0.07)	0.10	(0.14)

p < .05 **p < .01. ***p < .001.

Reference groups are Grade 5 in prior year, K8 schools, did not change schools during prior year, and so on. All covariates are grand mean centered.

The Robotics Summer Program was specifically designed to engage students who were not yet proficient in mathematics. Given the program’s focus, we also pre-specified a subgroup analysis to estimate the effects of the 2012 Robotics Summer Program on low-performing students. Roughly a third (36%) of the treatment students scored “basic” in mathematics on the state assessment in the spring prior to the summer program. These 60 treatment students and their 167 matches from the comparison group were included in the subgroup analyses. A formal baseline equivalence test compared the prior attendance rates of treatment and comparison students in this low-performing subsample. The two-level random intercept HLM model took account of students’ grade level and nesting in summer program sites (Table 3). The estimated difference between treatment and control means in the model was 0.42, a small non-significant prior attendance advantage for the control group (Hedges’s g = −.08). The groups were extremely closely matched.

Having established baseline equivalence in prior attendance, we then tested for program impacts on subsequent student attendance rates using our two-level random intercept model. As shown in Table 4, the intercept (adjusted mean 2012-2013 attendance rate in the subsample’s matched control group) was 93.8. The adjusted mean attendance rate of the treatment students in the subsample was 2.6 percentage points higher, 96.4%. This impact was both statistically significant, t(206) = 2.865, p = .005, and large enough to be educationally meaningful, with a Hedges’s g effect size of .37. Another way of stating the impact is that treatment students in the subsample attended, on average, about a week more of school than did the control students in the subsample (i.e., attended 4.7 days more during the course of the 180-day school year.)

We also conducted post hoc analyses using the other summer enrichment program students with prior attendance rate data (n = 407) as an alternative comparison group for all robotics students with prior attendance data (n = 188). Robotics students had a significantly higher program attendance rate (66%) than students in the other summer enrichment activities (59%). Using parallel models to those described above, we tested for baseline equivalence in prior year’s attendance between the groups. The groups were closely matched in prior attendance (differing by 0.12 percentage points). This translated into a Hedges’s g effect size of .02. Having established baseline equivalence in attendance, we then conducted impact analyses using a similar two-level HLM model. Robotics students had somewhat higher attendance rates the following year (96.5%) than did students in other enrichment activities (95.7%). This effect nearly met significance thresholds, t(589) = 1.78, p = .075, with a Hedges’s g effect size of .14.

Parallel analyses on the subsample of low-performing students in mathematics enrolled in the robotics program (n = 66) and in the other enrichment programs (n = 183) also established sufficient baseline equivalence in prior attendance rates between the groups (Hedges’s g = −.17),³ although robotics students had prior attendance rates of 0.85 percentage points lower than the comparison group. After the program, robotics students had attendance rates of 1.32 percentage points higher the following year than did students in other enrichment activities, controlling for attendance rates during the previous school year. This effect nearly met significance thresholds, t(243) = 1.67, p = .096, with a Hedges’s g effect size of .21.

Discussion

Although the impact on student achievement in mathematics that program developers had hoped to achieve was not detected in the first year of implementation, the program’s impact on student attendance rates in the year following the program is an important finding. Compared with students who had the very same attendance rate prior to the intervention, students who attended the Robotics Summer Program had significantly higher attendance rates at school in the year after the program. The effect size was even larger for students with low math achievement. These effects were found using both the matched comparison group with no summer school and the summer school students who did not participate in robotics, though the effects were not as strong for students with another potentially high-interest elective activity. Although we acknowledge that we cannot completely rule out possible selection bias in this quasi-experimental study and that a random assignment study would provide even more compelling tests of the program’s effects, the close matching of students on the most important predictor of attendance (prior attendance) as well as other demographic and school characteristics provided a strong research design from which to conclude that the program had an impact on students’ subsequent attendance.

The STEM Robotics Summer Learning Program provided students with the hands-on opportunity to construct, program, and operate a robot in competitions. This seems to have produced the envisioned increases in student engagement as measured by both program attendance and school attendance the following year. Although the study did not include qualitative interview data with treatment students that could have illumined the specific mechanisms underlying this attendance effect, the informal observations of program classrooms conducted by the research team found high levels of student engagement in the process of building robots, and a sense of accomplishment in student demeanors as they operated robots and participated in both informal and formal competitions. These observations were consistent with potential growth or maintenance of the components of an academic mind-set related to academic behaviors such as attendance, identified by Farrington et al. (2012): feelings of competence and success, belief in the value of effort to increase competence, and perceptions of the value of the task. Future research may be able to measure the specific intervening attitudes and determine their relationship to the behavioral outcome variable of school attendance.

It is important to note that the program had a significantly positive effect for males as well as for females. Our informal classroom observations found boys deeply engaged in building robots. These same boys are generally not as engaged in the regular academic classroom environment (Jacob, 2002). Given the widely discussed educational challenges of African American males in American society (e.g., Holzman, 2010; Lewis, Simon, Uzzell, Horwitz, & Casserly, 2010), this study’s finding of a program effect on their school engagement is an important contribution. Although research has documented a gender gap in “noncognitive” academic behaviors (Jacob, 2002) as well as the lower academic achievement of American males that has contributed to the gender gap in college attendance (DiPrete & Buchmann, 2013), there has been little attention to the gender gap in school attendance rates except in discussions of the higher rates of suspension among African American males (Lewin, 2012). Keeping boys interested in academic pursuits remains a challenge, particularly in high-poverty inner city contexts. When activities such as robotics engage them actively in building something complex that they can then manipulate and enjoy, they can see tangible results of their efforts, take pride in their competence, and gain a vision for how STEM and what happens in school can be relevant in the present and for their future.

The findings of this study emphasize the importance of investigating the potential impact of out-of-school programs on school-focused engagement. As Lawson and Lawson (2013) argued, research on school engagement needs to move beyond the traditional classroom and school to include out-of-school and community-focused activities. Activities outside of the regular school schedule can potentially build developmental competencies—particularly feelings of confidence, competence, and connection—that can keep students attached enough to school through attendance to increase their likelihood of success after leaving high school. Further research on the impact of similar programs on student engagement measures as well as on academic achievement measures will be a useful investment.

“Motivating the academically unmotivated” is one of the critical issues of the 21st century, as Hidi and Harackiewicz (2000, p. 151) have argued. Finding ways to stir up student interest in pursuing learning activities to maintain even the crudest indicator of engagement, simple school attendance, remains a challenge for most high-poverty secondary schools. As Ainley (2012) advocated, there is a need for more focused theoretical attention to the role of “interest” in student motivation and engagement. In particular, how can schools and other organizations focused on youth development both trigger and maintain situational interest so that students begin to internalize interests leading to increased motivation and engagement? Building on the concept of “authentic work” (Newmann, King, & Carmichael, 2007) as well as the research of Deci (1998) and Renninger (2000), Skinner and Pitzer (2012) argued that “active participation, engagement, and effort are promoted by tasks that are hands-on, heads-on, project-based, relevant, progressive, and integrated across subject matter, or in other words, intrinsically motivating, inherently interesting, and fun” (pp. 28-29). As Hug, Krajcik, and Marx (2005) argued, “using innovative technologies to promote learning and engagement” is a promising way to encourage science learning in urban settings (p. 446). The results of our study suggest that continued investment in high-interest elective activities such as robotics could have a significant impact on helping students remain engaged in school.

This study contributes to advancing knowledge in urban education in several ways. It addresses some of the “gaps” noted by Milner and Lomotey (2014). Situating the study within the theoretical framework of motivation and student interest addresses psychological factors that shape urban student experiences. It also narrows the “theoretical gap” and illuminates structural underpinnings of the observed achievement gaps, as the behavioral manifestation of engagement—attendance—is related to both neighborhood and family experiences (sociological factors) and educational practices within schools (curriculum and instruction). Recognizing the critical challenge of attendance and of student engagement more generally is crucial for urban education research as it seeks solutions for the observed achievement gaps. This study address the “theory to practice gap” in its focus on how a particular hands-on activity such as robotics can potentially help increase urban students’ sense of competence, interest in learning, and overall engagement with school. The results of this study of a robotics program highlight a potentially effective way to increase student engagement in urban schools, which is likely to have a positive impact on academic outcomes. Given the large majority of African American males among program participants, the study’s findings echo Noguera’s (2014) emphasis on the importance of engaging out-of-school time programs for Black males. Building on the emerging research frameworks for urban education, this study contributes to the evidence regarding promising strategies for improving the educational experiences of urban students and their academic outcomes.

Footnotes

Authors’ Note

The authors wish to thank Bruce Randel and anonymous reviewers for their helpful comments.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: The authors received financial support from a local education foundation (unnamed to protect the anonymity of the district) for the research on which this article is based.

Notes

Author Biographies

Martha Abele Mac Iver is an associate research professor in the School of Education at Johns Hopkins University. Her research has included numerous evaluations of interventions designed to improve school outcomes for historically underserved students. She is currently involved in developing and testing new interventions to improve high school student success using a continuous improvement framework.

Douglas J. Mac Iver is a research professor in the School of Education at Johns Hopkins University. He is a developmental psychologist with expertise in whole-school reform and in the multilevel modeling of educational achievement and motivation in the middle and secondary grades. His research focuses on the impact of instructional programs and disengagement prevention programs on students’ achievement and engagement.

References

Ainley

(2012). Students’ interest and engagement in classroom activities. In Christenson

Reschly

Wylie

(Eds.), Handbook of research on student engagement (pp. 283-302). New York, NY: Springer.

Allensworth

Easton

(2007). What matters for staying on-track and graduating in Chicago public high schools. Chicago, IL: Consortium on Chicago School Research.

Balfanz

(2009). Putting middle grades students on the graduation path. Baltimore, MD: Johns Hopkins University Everyone Graduates Center.

Balfanz

Byrnes

(2012). The importance of being in school: A report on absenteeism in the nation’s public schools. Baltimore, MD: Johns Hopkins University School of Education Everyone Graduates Center.

Balfanz

Byrnes

(2013). Meeting the challenge of combating chronic absenteeism: Impact of the NYC Mayor’s Interagency Task Force on Chronic Absenteeism and School Attendance and its implications for other cities. Baltimore, MD: Johns Hopkins University School of Education Everyone Graduates Center.

Balfanz

Herzog

Mac Iver

D. J.

(2007). Preventing student disengagement and keeping students on the graduation path in urban middle-grades schools: Early identification and effective interventions. Educational Psychologist, 42, 223-235.

Beer

Chiel

Drushel

(1999). Using autonomous robotics to teach science and engineering. Communications of the ACM, 42, 85-92.

Benitti

(2012). Exploring the educational potential of robotics in schools: A systematic review. Computers & Education, 58, 978-988.

Bloom

Thompson

S. L.

Unterman

(2010). Transforming the high school experience: How New York City’s new small schools are boosting student achievement and graduation rates. New York, NY: MDRC.

10.

Bloom

Unterman

(2013). Sustained progress: New findings about the effectiveness and operation of small public high schools of choice in New York City. New York, NY: MDRC.

11.

Chang

H. N.

Romero

(2008). Present, engaged, and accounted for: The critical importance of addressing chronic absence in the early grades. New York, NY: National Center for Children in Poverty.

12.

Clement

Gwynne

Younkin

(2001). Attendance waivers evaluation report. Fort Lauderdale, FL: School Board of Broward County.

13.

Connolly

Olson

L. S.

(2012). Early elementary performance and attendance in Baltimore City Schools’ pre-kindergarten and kindergarten. Baltimore, MD: Baltimore Education Research Consortium.

14.

Cooper

Charlton

Valentine

J. C.

Muhlenbruck

(2000). Making the most of summer school: A meta-analytic and narrative review. Monographs of the Society for Research in Child Development, 65(Serial No. 260), 1-118.

15.

Corville-Smith

Ryan

B. A.

Adams

G. R.

Dalicandro

(1998). Distinguishing absentee students from regular attenders: The combined influence of personal, family, and school factors. Journal of Youth and Adolescence, 27, 629-640.

16.

Coxon

(2012). The malleability of spatial ability under FIRST LEGO league-based robotics simulation. Journal for the Education of the Gifted, 35, 291-316.

17.

Credé

Roch

S. G.

Kieszczynka

U. M.

(2010). Class attendance in college: A meta-analytic review of the relationship of class attendance with grades and student characteristics. Review of Educational Research, 80, 272-295.

18.

Deci

E. L.

(1998). The relation of interest to motivation and human needs: The self-determination theory viewpoint. In Hoffman

Krapp

Renninger

K. A.

Baumert

(Eds.), Interest and learning (pp. 146-162). Kiel, Germany: Institute for Science Education.

19.

DiPrete

Buchmann

(2013). The rise of women: The growing gender gap in education and what it means for American schools. New York, NY: Russell Sage.

20.

Dong

Maynard

R. A.

(2013). PowerUp!: A tool for calculating minimum detectable effect sizes and sample size requirements for experimental and quasi-experimental designs. Journal of Research on Educational Effectiveness, 6, 24-67.

21.

Dynarski

James-Burdumy

Moore

Rosenberg

Deke

Mansfield

(2004). When schools stay open late: The national evaluation of the 21st Century Community Learning Centers Program—New findings. Washington, DC: U.S. Department of Education, National Center for Education Evaluation and Regional Assistance.

22.

Eccles

J. S.

(June, 2008). Can middle school reform increase high school graduation rates? (California Dropout Research Policy Brief 12). Santa Barbara: University of California, Santa Barbara.

23.

Eccles

J. S.

Midgley

(1989). Stage-environment fit: Developmentally appropriate classrooms for young adolescents. Research on Motivation in Education, 3, 139-186.

24.

Farrington

C. A.

Roderick

Allensworth

Nagaoka

Keyes

T. S.

Johnson

D. W.

Beechum

N. O.

(2012). Teaching adolescents to become learners: The role of noncognitive factors in shaping school performance. Chicago, IL: University of Chicago Consortium on Chicago School Research.

25.

Fine

(1991). Framing dropouts: Notes on the politics of an urban public high school. Albany: State University of New York Press.

26.

Finn

J. D.

(1989). Withdrawing from school. Review of Educational Research, 59, 117-142.

27.

Fredricks

J. A.

Blumenfeld

P. C.

Paris

A. H.

(2004). School engagement: Potential of the concept, state of the evidence. Review of Educational Research, 74, 59-109.

28.

Gottfredson

Cross

A. B.

Wilson

Rorie

Connell

(2010). Effects of participation in after-school programs for middle school students: A randomized trial. Journal of Research on Educational Effectiveness, 3, 282-313.

29.

Gottfried

M. A.

(2010). Evaluating the relationship between student attendance and achievement in urban elementary and middle schools: An instrumental variables approach. American Educational Research Journal, 47, 434-465.

30.

Gottfried

M. A.

(2011a). The detrimental effects of missing school: Evidence from urban siblings. American Journal of Education, 117, 147-182.

31.

Gottfried

M. A.

(2011b). Absent peers in elementary years: The negative classroom effects of unexcused absences on standardized testing outcomes. Teachers College Record, 113, 1597-1632.

32.

Gottfried

M. A.

(2013). Quantifying the consequences of missing school: Linking school nurses to student absences to standardized achievement. Teachers College Record, 115, 1-30.

33.

Grubbs

(2013). Robotics intrigue middle school students and build STEM skills. Technology and Engineering Teacher, 72, 12-16.

34.

Hayes

(2006). A primer on multilevel modeling. Human Communication Research, 32, 385-410.

35.

Hidi

Harackiewicz

J. M.

(2000). Motivating the academically unmotivated: A critical issue for the 21st century. Review of Educational Research, 70, 151-179.

36.

Holzman

(2010). Yes we can: The Schott 50 state report on public education and black males. Cambridge, MA: Schott Foundation for Public Education.

37.

Hug

Krajcik

Marx

(2005). Using innovative learning technologies to promote learning and engagement in an urban science classroom. Urban Education, 40, 446-472.

38.

Jacob

B. A.

(2002). Where the boys aren’t: Noncognitive skills, returns to school and the gender gap in higher education. Economics of Education Review, 21, 589-598.

39.

Lan

Lanthier

(2003). Changes in students’ academic performance and perceptions of school and self before dropping out of schools. Journal of Education for Students Placed at Risk, 8, 309-332.

40.

Lauer

Akiba

Wilkerson

Apthorp

Snow

Martin-Glenn

(2006). Out-of-school-time programs: A meta-analysis of effects for at-risk students. Review of Educational Research, 76, 275-313.

41.

Lauver

S. C.

(2002). Assessing the benefits of an after-school program for urban youth: An impact and process evaluation. Dissertations available from ProQuest. (Paper AAI3043903). Retrieved from http://repository.upenn.edu/dissertations/AAI3043903

42.

Lawson

(2013). New conceptual frameworks for student engagement research, policy and practice. Review of Educational Research, 83, 432-479.

43.

Lewin

(2012, March 6). Black students face more discipline, data suggests. New York Times. Retrieved from http://www.nytimes.com/2012/03/06/education/black-students-face-more-harsh-discipline-data-shows.html?_r=0

44.

Lewis

Simon

Uzzell

Horwitz

Casserly

(2010). A call for change: The social and educational factors contributing to the outcomes of black males in urban schools. Washington, DC: Council of the Great City Schools.

45.

Mac Iver

. (2007). What reform left behind: A decade of change at one urban high school. Education and Urban Society, 40, 3-35.

46.

Mac Iver

. (2011). The challenge of improving urban high school graduation outcomes: Findings from a randomized study of dropout prevention efforts. Journal of Education for Students Placed at Risk, 16, 167-184.

47.

Mac Iver

Messel

. (2012). Predicting high school outcomes in the Baltimore City Schools. Washington, DC: Council of the Great City Schools.

48.

Mac Iver

Plank

S. B.

Durham

Farley-Ripple

Balfanz

. (2008). The challenge of on-time arrival: The seven-year flight paths of Baltimore’s sixth graders of 1999-2000 (Research Report). Baltimore, MD: Johns Hopkins University.

49.

Marks

H. M.

(2000). Student engagement in instructional activity: Patterns in the elementary, middle, and high school years. American Educational Research Journal, 37, 153-184.

50.

Marvul

(2012). If you build it, they will come: A successful truancy intervention program in a small high school. Urban Education, 47, 144-169.

51.

Maryland State Department of Education. (2015). 2015 Maryland Report Card. Baltimore City Attendance Rate. Retrieved from http://reportcard.msde.maryland.gov/AttendanceRate.aspx?PV=183:M:30:AAAA:1:N:0:13:1:1:0:1:1:1:3

52.

Mauch

(2001). Using technological innovation to improve the problem solving skills of middle school students: Educators’ experiences with the LEGO Mindstorms robotic invention system. The Clearing House, 74, 211-213.

53.

McCombs

Augustine

Schwartz

Bodilly

McInnis

Lichter

Cross

A. B.

(2011). Making summer count. RAND Corporation. Retrieved from http://www.rand.org/pubs/monographs/MG1120.html

54.

Meece

(2003). Applying learner-centered principles to middle school education. Theory Into Practice, 42, 109-116.

55.

Milner

H. R.

IV Lomotey

K. H. R.

(2014). Introduction. In Milner

H. R.

Lomotey

(Eds.), Handbook of urban education (pp. xv-xxiv). New York, NY: Routledge.

56.

Neild

R. C.

Balfanz

(2006a). An extreme degree of difficulty: The educational demographics of urban neighborhood high schools. Journal of Education for Students Placed at Risk, 11, 131-141.

57.

Neild

R. C.

Balfanz

(2006b). Unfulfilled promise: The dimensions and characteristics of Philadelphia’s dropout crisis, 2000-2005. Philadelphia, PA: Philadelphia Youth Transitions Collaborative.

58.

Newmann

F. M.

King

M. B.

Carmichael

D. L.

(2007). Authentic instruction and assessment: Common standards for rigor and relevance in teaching academic subjects. Des Moines: Iowa Department of Education.

59.

Newmann

F. M.

Wehlage

C. G.

Lamborn

S. D.

(1992). The significance and sources of student engagement. In Newmann

F. M.

(Ed.), Student engagement and achievement in American secondary schools (pp. 11-39). New York, NY: Teachers College Press.

60.

Noguera

P. A.

(2014). Urban schools and the black male “challenge.” In Milner IV

H. R.

Lomotey

(Eds.), Handbook of urban education (pp. 114-128). New York, NY: Routledge.

61.

Renninger

(2000). Individual interest and its implications for understanding intrinsic motivation. In Sansone

Harackiewicz

J. M.

(Eds.), Intrinsic and extrinsic motivation: The search for optimal motivation and performance (pp. 373-404). San Diego, CA: Academic Press.

62.

Roderick

Camburn

(1999). Risk and recovery from course failure in the early years of high school. American Educational Research Journal, 36, 303-344.

63.

Rubin

D. B.

Thomas

(2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95, 573-585.

64.

Sheldon

S. B.

Epstein

J. L.

(2002). Improving student behavior and school discipline with family and community involvement. Education and Urban Society, 35, 4-26.

65.

Sinclair

M. F.

Christenson

S. L.

Thurlow

M. L.

(2005). Promoting school completion of urban secondary youth with emotional or behavioral disabilities. Exceptional Children, 71, 465-482.

66.

Skinner

Pitzer

(2012). Developmental dynamics of students’ engagement, coping, and everyday resilience. In Christenson

Reschly

Wylie

(Eds.), Handbook of research on student engagement (pp. 21-44). New York, NY: Springer.

67.

Stormshak

E. A.

Connell

Dishion

T. J.

(2009). An adaptive approach to family-centered intervention in schools: Linking intervention engagement to academic outcomes in middle and high school. Prevention Science, 10, 221-235.

68.

Wagstaff

Combs

Jarvis

(2000). Solving high school attendance problems: A case study. Journal of At-Risk Issues, 7, 21-30.

69.

Welch

Huffman

(2011). The effect of robotics competitions on high school students’ attitudes towards science. School Science and Mathematics, 111, 416-424.