Abstract
Lockdown drills are routinely conducted in schools across the U.S., yet little is known about the impact of such practices on participants—particularly for the faculty, staff, and administrators charged with student safety. The present study considers the effects of lockdown drills and associated emergency response training on perceived safety and preparedness at school for a sample of 3,000 school-based personnel in a large urban school district. After participating in drills and training, significant gains were made in perceived emergency preparedness, particularly among faculty and staff members, while perceived school safety—which already was particularly high—remained unaffected compared to ratings at baseline. Additionally, significant differences were found based upon respondent demographics, highlighting the importance in assessing subgroups’ needs during planning and implementation. Implications for school administration and relevant policymakers related to lockdown drills and training are considered within the context of creating a culture of preparedness.
Keywords
School safety always has been a priority for policymakers and the public alike, although after high profile mass shootings in educational institutions, the question of how to keep students, faculty, and staff safe often is raised with renewed focus (Madfis, 2016). Following the February 14, 2018 shooting at Marjory Stoneman Douglas High School in Parkland, FL, for example, the national discourse centered, in part, on emergency preparedness training, including lockdown drills, after it was revealed that a lack of prior training may have contributed to the lethality of the attack − teachers had received little instruction and students had received none regarding how to respond in an active shooter situation like the one that occurred (O’Matz, 2018). Lockdown drills, however, have been a staple in U.S. schools since the April 20, 1999 shooting at Columbine High School in Jefferson County, CO (Rygg, 2015; Schildkraut & Muschert, 2019). In fact, during the 2015 to 2016 academic year, 95% of U.S. public schools conducted lockdown drills; additionally, 92% practiced evacuation procedures (e.g., fire drills) and 76% ran shelter-in-place drills (Musu-Gillette et al., 2019).
Preparedness for emergencies, disasters, and crises focus on changing behaviors to reduce people’s risk of harm, injury, and/or death, and commonly are grounded in theoretical frameworks that focus on motivation to engage in protective behaviors based on their cognitive appraisal of the threat and the benefits of employing the action to reduce risk (Ejeta et al., 2015; Floyd et al., 2000). According to Protection Motivation Theory (Floyd et al., 2000), people evaluate potential threats in relation to how likely they think it is to occur (i.e., vulnerability, risk) and the potential consequences (i.e., severity). This is weighed against evaluations of the benefits of engaging in protective behavior, specifically whether the action(s) will be effective in reducing or eliminating the threat (i.e., response efficacy) and the person’s belief that they have the ability to take the actions needed to protect themselves and others (i.e., self-efficacy; Floyd et al., 2000). Although the current study does not test Protection Motivation Theory in its entirety, it assesses both perceptions of safety (relevant to the evaluation of threats) and emergency preparedness for school staff (i.e., knowing what to do to protect themselves and others) among faculty, staff, and administrators in K-12 schools. More specifically, this study examines whether engaging in lockdown drills and participating in training on an emergency response protocol may lead to increased perceptions of preparedness without adversely impacting perceptions of school safety.
Research on Lockdown Drills
Despite the widespread use of lockdown drills, there are two significant areas of concern related to these practices that warrant further attention. First, although lockdown drills are designed to prepare faculty, staff, and students for a range of school-based emergencies, active shooters being just one (Gubiotti, 2015; National Association of School Psychologists [NASP], 2018), there is no national standard for how these practices should be conducted (see, e.g., U.S. Government Accountability Office, 2007). Although the U.S. Department of Education (DOE, 2013) has offered guidance for schools related to emergency preparedness, the decisions of how many drills are required annually or the manner in which they are conducted typically fall to the individual states or, in some instances, schools or districts themselves. As a result, methods for carrying out practices for emergencies more broadly run the gamut, from all hazards plans to options-based active shooter-specific protocols, and from simple table-top exercises to full-scale exercises using simunition (non-lethal training ammunition; see Zraick, 2019). The latter has raised significant concerns, particularly after teachers in Indiana were injured after being shot with plastic pellets during a training simulation (Herron, 2019).
The second area of concern related to lockdown drills is the alarming absence of empirical research that is necessary to understand the impact of these practices on the individuals who participate in them (Schildkraut et al., 2020). Outcomes that are important to examine related to lockdown drills are the extent to which they impact perceptions of safety as well as actual and perceived emergency preparedness. Many studies that consider any perceptions of safety and/or preparedness related to lockdown or active shooter drills, however, focus on college or university students and do not assess these constructs in conjunction with actual participation in drills (e.g., Baer et al., 2014; Huskey & Connell, 2020; Peterson et al., 2015). There are only a handful of empirical studies that consider the impact of drills on perceived safety and emergency preparedness in the K-12 setting, and these focus on students (Dickson & Vargo, 2017; Jonson, Moon, Gialopsos, 2020; Schildkraut & Nickerson, 2020; Schildkraut et al., 2020; Zhe & Nickerson, 2007). Results have been mixed regarding perceptions of safety, with some researchers finding that participation in drills does not impact students’ perceptions of safety or fears and/or anxiety relative to comparison activities (Jonson, Moon, Hendry, 2020; Zhe & Nickerson, 2007), and others finding that students were less likely to report feeling safe in certain areas of their school after the practice than before (Schildkraut et al., 2020). In terms of preparedness, research indicates that drills can result in students, even in elementary school, being able to perform the skills necessary in a lockdown (Dickson & Vargo, 2017; Schildkraut & Nickerson, 2020; Zhe & Nickerson, 2007). To date, even as the body of research continues to expand, the majority of these studies focus on the impact of these practices on students, particularly as claims that active shooter and lockdown drills are traumatizing them circulate the public discourse (see, e.g., Everytown for Gun Safety, 2020).
Largely absent the line of inquiry is how such practices impact faculty, staff, and administrators—the very individuals charged with keeping students safe. In fact, to date, just one published study (Perkins, 2018) and several theses and doctoral dissertations (Bergh, 2009; Embry-Martin, 2017; Rider, 2015; Worthington, 2020) offer any insight into the impact of lockdown drills on the adult members of the academic community. 1 Despite variation in the locations where these studies were conducted and their respective methodologies (qualitative vs. quantitative, sample size), their findings consistently highlight important considerations for emergency preparedness in schools. Although nearly half of the respondents across these studies indicated they participated in drills, a lesser proportion reported that they had received any related training (Embry-Martin, 2017; Perkins, 2018) or that the instruction they had received was adequate to be able to respond effectively (Bergh, 2009; Rider, 2015). In Bergh’s (2009) study, for example, 75% of participants indicated that more training, including different scenarios, was needed, which was particularly important given that none of the schools utilized the same emergency response protocol (see also Perkins, 2018, who highlights the need for consistency among practices) and the majority of the respondents could not correctly remember all of the steps in the lockdown. Embry-Martin’s (2017) findings also highlighted the importance of the training component, in that reported self-efficacy among teachers increased after receiving instruction of how to respond. Similarly, Worthington’s (2020) study indicated that individuals who participated in active shooter training reported less anxiety and more preparedness compared to those who had not received such training. Although important first steps in better understanding the role of school employees in emergency response, these studies employed cross-sectional designs not equipped to fully assess a process that unfolds over time.
Although studies assessing school faculty, staff, and administrator perceptions of school safety and emergency preparedness in relation to lockdown drills are very limited, there is some related research about K-12 school staff perceptions and practices that may offer insight into these issues, particularly with regard to demographic differences. For example, Kano et al.’s (2007) study found that school administrative staff reported perceiving lower levels of preparedness compared with classified or certificated staff. Although the authors stated they did not know what specifically contributed to this disparity, they noted that a person’s position in the school may impact perceptions and reporting about emergencies (Kano et al., 2007). In relation to sex differences, studies of public health/household disaster preparedness within the U.S. population have indicated that females report being less prepared than males (Ekenga & Ziyu, 2019).
Lockdown drills emerged in response to high-profile yet statistically rare school shootings (Schildkraut & Muschert, 2019) and subsequently do not discriminate based on racial differences, unlike other policies such as zero tolerance or varying forms of school-based punishment (e.g., Fenning & Rose, 2007; Hoffman, 2014; Skiba et al., 2011). Broader research on emergency preparedness and school safety, however, may provide insight into why perceptions of safety practices such as lockdown drills may differ across racial/ethnic groups. Regarding personal/household disaster preparedness, Black and Hispanic respondents have reported being less prepared than White respondents (Bethel et al., 2013). Relevant to school emergency preparedness specifically, studies have found that public schools with higher proportions of students from racial and ethnic minority groups had fewer automated external defibrillators (AEDs; White et al., 2016) and that staff in these schools had inadequate training in using these devices (Saberian et al., 2018). In terms of perceptions of school safety, a survey of the general public found that Black respondents were more likely than individuals from other racial and ethnic groups to perceive schools as less safe (Shelley et al., 2017). Similarly, there is a substantive body of literature about race, ethnicity, and school safety, particularly with regard to the finding that harsh and exclusionary discipline practices (e.g., zero tolerance) are more likely in schools with higher proportion of Black students (Fenning & Rose, 2007; Fergus et al., 2014; Payne & Welch, 2010; Welch & Payne, 2010). Despite this context and relevant research on demographic differences in emergency preparedness and school safety, these issues rarely have been examined with regard to educators’ perceptions of safety and emergency preparedness; thus, they were explored in this study.
Current Study
The overarching goal of the present study was to fill this gap by better understanding how faculty, staff, and administrators perceive safety and preparedness in relation to lockdown drills and associated training. To date, the few studies that have been conducted assessing any perceptions of these groups related to lockdown drills have utilized a cross-sectional design. The present study expanded on this by considering changes in perceptions of participants as a function of different drill conditions (e.g., both prior to and after training) with the inclusion of a pre-test component for the purpose of providing a baseline understanding of such attitudes. Using a sample of school-level personnel in a large urban district in Central New York, we analyzed data collected at three different time points (pre-test, post-drill prior to training, post-training drill) to answer the following research questions:
What impact, if any, do lockdown drills and training have on feelings of school safety among faculty, staff, and administrators?
What impact, if any, do lockdown drills and training have on perceptions of emergency preparedness among school faculty, staff, and administrators?
We hypothesized that participation in lockdown drills, and particularly receiving training and participating in the drills would not impact feelings of school safety, as has been found in studies with students (Jonson, Moon, Gialopsos, 2020; Zhe & Nickerson, 2007). Additionally, consistent with prior research (Dickson & Vargo, 2017; Schildkraut & Nickerson, 2020; Schildkraut et al., 2020; Worthington, 2020; Zhe & Nickerson, 2007), we expected that participation in the training and drill would increase perceptions of emergency preparedness. Further, as a supplementary exploratory analysis, we examined whether demographic characteristics of school personnel predicted differences in perceptions of school safety and emergency preparedness.
The present study seeks to answer the research questions in conjunction with a specific emergency preparedness program—the Standard Response Protocol-Extended Edition (SRP-X) from the I Love U Guys Foundation (Keyes & Deffner, 2015). Currently utilized in more than 30,000 schools nationwide, SRP-X incorporates language consistent with that of the National Incident Management System (NIMS) from the Federal Emergency Management Agency (FEMA), utilized by first responders in an effort to standardize communications during a crisis (U.S. Department of Homeland Security, 2012). Additionally, SRP-X is an all hazards plan and includes operational guidance for lockouts (used for threats near the school grounds), evacuations (e.g., fire drills), sheltering in place (typically used for natural disasters), and holding in place (used when the hallways need to be kept clear), in addition to lockdowns (Keyes & Deffner, 2015), as recommended by federal guidance (U.S. Department of Education [DOE], 2013). Moreover, this protocol mirrors New York’s Education Department requirements that educational institutions be prepared for these five situations (Bakst, 2015), meaning that implementing this program in schools ensures their greater compliance with state mandates.
Method
Project Overview
The survey data utilized in the present study were collected as part of a larger initiative in the school district to implement a new emergency preparedness protocol. The district, comprised of 30 school buildings, serves over 3,500 faculty and staff members in addition to more than 21,000 students. 2 Following the Parkland shooting, the district’s Department of Public Safety (DPS) partnered with the researchers to standardize response plans across schools. This became all the more important as an initial survey of building principals during the project’s planning stages revealed that there was a lack of consistency in protocols being used across the district. In other words, although each school was complying with state requirements to conduct four lockdown drills (in addition to eight fire drills) annually (see New York State Education Law, Title 1, Article 17 § 807), the manner in which each practice was conducted differed by building due to a lack of training and instructional guidance on how they should be carried out.
An initial baseline survey was conducted in September 2018 to better understand faculty, staff, and administrators’ feelings of safety and preparedness in their schools. Between October and November 2018, each school participated in an unscheduled lockdown drill, conducted with the assistance of DPS and school-based security officers. Although the drills were largely unscheduled (schools were notified about a week in advance that the research team would be coming, though they were not provided the exact date and time to elicit the most accurate responses), they were never unannounced in accordance with guidance from the National Association of School Psychologists (NASP, 2017, 2018). Specifically, each time a practice was initiated, the call was accompanied by the phrase “this is a drill,” which was subsequently repeated.
Each drill took between 7 and 17 minutes to complete, based on the size of the building. After the drill was initiated, members of the research team checked each room to assess four specific criteria: (1) if the doors were locked, (2) if the lights were off, (3) if occupants could be seen or heard from the hallway, and (4) if there was any response from the room to a door knock. The researchers then entered the room to check if it was vacant, identified themselves to occupants, and reminded them to remain in lockdown until they were given the all clear by the school’s principal. Once all of the rooms had been checked, a short debrief period was called to allow teachers and their students, as well as administrators and staff, to discuss the drill and also compile any questions they had for the researchers. After several minutes, the drill was concluded and normal activity resumed. In addition to being consistent with the SRP-X protocol’s operational guidance, these steps also mirror NASP’s (2018) recommendations for mitigating trauma during lockdowns.
Approximately 1 week after the drill (November 2018), the survey instrument was re-administered to assess what impact, if any, the drill had on faculty, staff, and administrators’ perceptions of safety and preparedness. Between January and March 2019, teachers and their students together took part in SRP-X training conducted by the research team; administrators and staff members also attended the sessions. Over 24 days, 70 individual training sessions were conducted within the 30 buildings. Following the training (March 2019), a second round of lockdown drills took place using the same procedures as the first. This allowed the researchers to assess if participating in the training yielded any improvement in how schools were responding based on the four assessment criteria (locks, lights, out of sight, door knock). Finally, the survey instrument was deployed for a third time (April 2019) to assess not only the impact of training but also participation in multiple drills.
Participants
Upon receiving institutional review board approval for the study from the principal investigator’s university, DPS provided the research team with an email list for all school-based personnel. The email addresses then were preloaded into Survey Monkey, which was used to administer the surveys electronically. Emails were sent to each of the potential participants outlining the study, inviting them to participate, and providing them with a unique link generated by Survey Monkey to internally track completion to complete the survey. 3 Follow-up reminders were sent 7 and 14 days after the initial invitation to anyone who had not yet participated, with the survey closing 3 weeks after its initial deployment. This same process was employed at all three time points and 3,221 faculty, staff, and administrators were invited to participate.
Table 1 presents the demographic distribution of respondents based on time the survey was taken. As indicated, each time’s sample was predominantly female and white. The mean ages of respondents were 43.72 years (T1), 43.74 years (T2), and 44.47 years (T3) and did not vary significantly based on when the survey was completed. The majority of respondents at all three time points identified themselves as either faculty or staff members. Finally, respondents were asked how long they had worked for the district. The average length of employment was 12.99 years (T1), 12.83 years (T2), and 13.14 years (T3) and also did not differ significantly based on time.
Distribution of Demographic Characteristics: Faculty, Staff, and Administration Respondents.
Note. Results presented as raw counts with frequency percentages in parentheses. Frequency percentages are based on the total number of surveys collected at each time and may not round to 100.0% due to missing data.
Measures
Each survey instrument included panels of questions aimed at capturing respondents’ perceptions of safety and preparedness. Related to safety, respondents were asked to rate their agreement to the statements “I feel safe” (1) at my school; (2) in my classroom(s); (3) in the hallways; and (4) outside on school grounds, which corresponds to general perceptions of school safety irrespective of the lockdown drills. Responses were recorded along a five-point Likert scale ranging from Strongly Disagree (1) to Strongly Agree (5), with higher mean scores indicating stronger feelings of safety. These questions then were summed into a scale measuring perceived school safety, which had a corresponding Cronbach’s alpha of .906, indicating high internal consistency among the questions included in the scale. Higher values on the scale indicated greater agreement with perceived school safety.
A second set of questions assessed respondents’ familiarity with the functional annexes of the SRP-X protocol. Using a five-point Likert scale, respondents were asked to rate their agreement to the statements “I know what to do” in each scenario: lockout, lockdown, evacuate, shelter, and hold. Responses then were summed to create an emergency preparedness scale with higher values corresponding to greater feelings of preparedness. The reliability analysis returned a Cronbach’s alpha of .898, again indicating high consistency among the included measures.
Results
Perceptions of Safety
The first research question considers whether participating in multiple lockdown drills and emergency preparedness training impacts faculty, staff, and administrators’ perceptions of school safety. To answer this question, One-way Analyses of Variance (ANOVAs) were used with the school safety scale included as the dependent measure and time (baseline, after the first lockdown drill, and following training and the second drill) serving as the independent variable. Scheffé’s post hoc analyses also were included to determine where differences between phases, if any, existed. The results of these analyses, both for the full sample and disaggregated by respondents’ position, are presented in Table 2. In first assessing the full sample, the results indicate that there are no significant differences in perceived school safety based on time. In other words, participating in lockdown drills and training did not significantly impact general perceptions of school safety among respondents. Further, this finding holds when perceptions are assessed based on respondents’ role within the school buildings. Regardless of whether the respondent serves as faculty, staff, or in another capacity (including administration), their perceived school safety is not significantly altered by participating in lockdown drills and training.
Means and Standard Deviations (SD) of Faculty/Staff/Administrator Perceptions of Safety by Time with One-Way ANOVA Statistics and Post hoc Analyses (N = 3,000).
Note. T1 = pre-test survey; T2 = post-drill survey; T3 = post-training/drill survey.
p < .01. **p < .001.
Perceptions of Preparedness
Consideration also was given to how perceived preparedness, both collectively and across SRP-X’s five emergency scenarios, may have varied as a result of lockdown drills and training. One-way ANOVAs again were used to assess potential disparities, with Scheffé’s post hoc analysis employed to determine where, when applicable, such differences existed. To protect from Type I error when conducting analyses on multiple dependent variables, we included a Bonferroni type adjustment (.05/n of comparisons, or .05/5 = .01), and used the more conservative p-value of <.01 for significance. The results for the full sample, as well as disaggregated by respondents’ role, across the three survey points are presented in Table 3.
Means and Standard Deviations (SD) of Faculty/Staff/Administrator Familiarity with SRP-X Annexes by Time with One-Way ANOVA Statistics and Post hoc Analyses (N = 3,000).
Note. T1 = pre-test survey; T2 = post-drill survey; T3 = post-training/drill survey.
p ≤ .01. **p ≤ .001.
Significant differences in perceived emergency preparedness were found based on time when first assessing the full sample. Specifically, respondents who completed the survey following the training and the second drill (Time 3) expressed feeling more prepared to respond to emergencies (F = .200, p < .001) as compared to those who were evaluated at the baseline (Time 1) or following the first drill but prior to training (Time 2). Moreover, as the results indicate, significant differences in respondents’ reported agreement that they felt prepared to respond varied significantly across all five scenarios. At Time 3, respondents were significantly more likely to agree that they knew how to respond during a lockout (F = 43.167, p < .001), lockdown (F = 8.674, p < .001), evacuation (F = 8.038, p < .001), shelter in place (F = 14.895, p < .001), and hold in place (F = 24.335, p < .001) than either of the other two survey time points.
When examining differences across perceptions disaggregated by respondent role, several interesting findings emerge. For faculty, statistically significant differences in average agreement were found for several of the scenarios. Respondents identifying as faculty members were significantly more likely to express agreement that they knew how to respond in a lockout (F = 19.931, p < .001) and hold in place (F = 10.223, p < .001) after completing training and the second lockdown drill (Time 3) than at the pretest (Time 1) or after the first drill (Time 2). 4 Significant differences by time, however, were not found related to the lockdown or evacuation scenarios.
For staff, on the other hand, statistically significant differences in perceived emergency preparedness were found across all five scenarios. As compared to the pretest (Time 1) and following the first drill (Time 2), respondents identifying as building staff were more likely to agree that they knew how to respond in a lockout (F = 21.316, p < .001), lockdown (F = 7.180, p < .001), shelter in place (F = 12.027, p < .001), and hold in place (F = 12.511, p < .001) after completing the training and participating in the second lockdown drill (Time 3). They also were more likely to agree that they know how to respond if an evacuation call was made after the training and second drill as compared to after just the first drill (F = 5.434, p < .001). Further, for both faculty (F = 10.529, p < .001) and staff (F = 14.674, p < .001), greater agreement related to emergency preparedness overall was found at the final survey point (Time 3) comparative to the other two intervals. Respondents identifying as administrators or occupying other roles in the schools, however, did not differ significantly in perceptions related to preparedness by time across any of the five emergency scenarios or emergency preparedness more broadly.
Predictors of Feelings of Safety and Preparedness
Exploratory supplemental analyses were performed to provide additional consideration as to what characteristics of respondents, if any, may predict differences in reported perceptions related to school safety and emergency preparedness; the OLS regression results are presented in Table 4. In first examining perceived school safety, the findings indicate that, as compared to females, males were more likely to express greater agreement that they feel safe at school (b = .425, p < .01). Age was found to be inversely correlated with school safety; as the age of the respondent increased, perceived school safety decreased (b = −.028, p < .001). Respondents who identified as Black were more likely than White respondents to report feeling safe (b = .455, p < .05). Finally, respondents who identify as administrators or in roles other than faculty or staff (b = 1.843, p < .001) were more likely to agree that they feel safe at school as compared to faculty members, whereas staff (b = −.307, p < .05) were less likely to express agreement.
OLS Regression Results for Perceived School Safety and Emergency Preparedness (N = 3,000).
Note. Full model with expanded “other race” category available upon request.
p ≤ .05. **p ≤ .01. ***p ≤ .001.
Several of the significant correlates of increased school safety also were found to predict greater levels of perceived emergency preparedness. Respondents identifying as male (b = .858, p < .001), Black (b = .513, p < .05), Hispanic (b = 1.476, p < .001), and administrators or in other roles (b = 1.250, p < .001) were significantly more likely to express greater agreement that they knew how to respond to an emergency at school as compared to females, Whites, and faculty, respectively. The longer the respondent had been employed by the district, the more likely they were to report knowing how to respond to an emergency (b = .025, p < .05). Finally, respondents completing the survey after training and the second drill were more likely to report greater perceived emergency preparedness as compared to those who completed it at the beginning of the project (b = 1.024, p < .001).
Discussion
For more than 20 years since Columbine, lockdown drills have been as much of a staple in the U.S. education system as reading, writing, and arithmetic. Yet despite their widespread use, there is an alarming dearth of research that is necessary to better understand their effects (Schildkraut & Nickerson, 2020; Schildkraut et al., 2020). Little is known about the impact of these practices on students, and even less is known about how they affect those tasked with keeping the students safe: faculty, staff, and administrators. The present study sought to fill this gap by not only considering how participation in lockdown drills impacts school-based employees’ perceptions of school safety generally and emergency preparedness specifically, but how such feelings potentially vary across multiple time points, including with the introduction of training.
Perceptions of School Safety
As the results of the present study indicate, general perceptions of school safety largely were not impacted by the introduction of either lockdown drill or emergency response training. In other words, participants expressed generally feeling safe at their school even when preparing for emergency situations by participating in multiple lockdown drills and training. This finding was consistent both for the full sample as well as when disaggregated by respondents’ role (faculty, staff, or administrator/other). It bears noting, however, that perceived school safety already was high among these individuals and remained so across time. A comparison of means test reveals that at all three time points, the mean score on the collapsed scale was 3.99 (out of 5), representing agreement that respondents generally feel safe at school. 5 This is consistent with previous studies finding that participation in drills does not adversely impact students’ perceptions of safety or anxiety more than engagement in control group scenarios or other related preparedness activities (Zhe & Nickerson, 2007). It differs, however, from the findings from our previous work with the student sample in this district that showed some decreased feelings of safety after participating in multiple drills (Schildkraut et al., 2020).
Perceiving schools as safe is crucial for faculty, as well as staff and administrators, as researchers have found that such beliefs increase work engagement and decrease teacher burnout (see, e.g., Bass et al., 2016), both of which have considerable implications for students and their respective learning and success. In the context of lockdown drills and associated training, even perceptions of school safety more generally are particularly important in light of concerns about the harm that such practices can cause (Everytown for Gun Safety, 2020; Herron, 2019), including anxiety, depression, behavioral issues, and poor academic performance (Rygg, 2015). Best practices from NASP (2017, 2018) recommend, among other things, that educators model calm behavior in order to minimize the potential for students to experience trauma during a drill; those who perceive their school to be safer may be able to do this more successfully. Completing training in addition to the drills can serve to reinforce the skills necessary to be able to model this behavior by ensuring that participants know the precise actions to be taken. At the same time, faculty, staff, and administrators who perceive their school to be safe also may be able to successfully manage their own potentially adverse reactions to lockdown drills, recognizing that the practice is a routine preparation tool like a fire drill rather than an indicator that there is a reason for heightened concern. It is worth noting, however, that these results only can be generalized to drills and training that utilize best practice procedures (e.g., SRP-X; see Keyes & Deffner, 2015; NASP, 2017, 2018). The findings here do suggest, however, that school safety preparedness training and practices do not have to be traumatizing experiences for participants in order to be effective.
Perceptions of Emergency Preparedness
While perceived school safety remained consistent across the course of the project, the results indicate that feelings of preparedness improved significantly with the introduction of training and continued practice. Specifically, as compared to the initial pre-test and following the first drill, respondents who completed their survey after the training and second drill reported significantly greater perceptions of emergency preparedness. Such a finding highlights the importance of both training and practice in fostering a culture of preparedness. Within the context of Protection Motivation Theory (Floyd et al., 2000), our findings suggest that participation in drills and training increased educators’ preparedness, which is important in terms of self-efficacy and response efficacy, or believing that one can engage in a behavior to reduce or eliminate a threat. This also is consistent with research indicating that school personnel reported higher levels of perceived emergency preparedness when they engaged in related activities, such as having a copy of the school’s emergency plan and participating in drills (Kano et al., 2007). Research with students previously indicated that participation in lockdown drills increased perceptions of and/or ability to be prepared in an emergency (e.g., Dickson & Vargo, 2017; Schildkraut & Nickerson, 2020; Zhe & Nickerson, 2007), and the current study extends this research to educators as well. These increased perceptions of preparedness, along with the lack of adverse impact on perceptions of safety, interpreted within Protection Motivation Theory suggest that educators may be motivated to enact protective behaviors (i.e., implementing lockdown procedures) as they weigh the benefits in relation to the consequences.
In the context of school-based employees, the goals of training and drills are two-fold. First, training empowers faculty, staff, and administrators with the tools necessary to keep their students safe in times of crisis. Second, practicing their plans helps to build muscle memory (Shusterman, 2011), which is important because the body will do what it is trained to do even when cognitive functioning is impaired, such as with stress that may arise during an actual emergency. Accordingly, the present study’s findings suggest that participants are better equipped to respond when needed and also are more confident in being able to do so.
At the same time, differences based on the respondents’ role also highlight important considerations for emergency preparedness training. As indicated, participants identifying as staff members reported gains in perceived preparedness across all five scenarios, whereas faculty members only showed significant differences for lockout and hold. 6 Notably, while New York State requires schools to be prepared for all five scenarios, the mandate in regard to drills is that schools only are required to practice lockdowns and evacuations. 7 Therefore, for faculty, their exposure to training on scenarios they do not regularly practice may have contributed to improvements in their perceived abilities to respond to such situations. For staff, however, participating in training to inform them of how to respond in the different emergencies was likely invaluable given that oftentimes, they do not participate in the drills. The research team repeatedly observed across all 30 buildings, for example, that key school-based staff, including cafeteria workers, janitorial staff, and shipping clerks, among others, continued to work during both drills. Since staff in the current study were not practicing and getting hands-on experience, the training potentially may have at least begun to empower them for emergency response, though it is not a sufficient substitute to build muscle memory. This aligns with previous research, which found that administrative staff in schools reported lower perceptions of preparedness compared with classified or certificated staff (Kano et al., 2007); although they did not assess specific roles in the school, it is possible that those staff in support roles were not as actively involved in preparedness efforts, further highlighting the importance of including all school-based personnel in training and drills.
Factors Impacting Perceptions of Safety and Preparedness
Finally, through our supplementary exploratory analyses, we considered whether demographic characteristics potentially were correlated with perceptions of safety and emergency preparedness. The findings revealed that females expressed feeling less safe and prepared for emergencies compared to males. This is consistent with related research showing that female teachers report feeling less safe in schools than their male counterparts (Meyer et al., 2002), and that females also report less personal/household emergency preparedness than males (Ekenga & Ziyu, 2019).
Respondents identifying as Black and Hispanic reported feeling more prepared for emergencies relative to White respondents, but only Black educators reported comparatively increased perceptions of safety. The fact that Black and Hispanic respondents reported feeling more prepared for emergencies than their White peers may reflect their increased vulnerability to experiencing emergencies (Lemyre et al., 2009), although previous research has found that in terms of individual and household preparedness, Black and Hispanic respondents report being less prepared than their White counterparts (Bethel et al., 2013). Within the broader context of emergency preparedness in educational settings, working in schools with predominantly Black or Hispanic populations has been found to be a predictor of inadequate AED training for faculty and staff (Saberian et al., 2018). Results of the current study about Black and Hispanic educators feeling more prepared after the drill and training compared to their White colleagues may reflect that practice and training potentially help to reduce the racial/ethnic disparities found in related research.
More broadly, the role of sex, race/ethnicity, and their intersectionality with regard to perceptions of school safety and emergency preparedness is not well understood and should be examined further in future research. Still, our findings suggest that, as districts continue to implement new emergency preparedness protocols or revise existing plans, efforts may be made to tailor materials and training to subgroups based on their specific needs (see, generally, Hughes et al., 2015). By accounting for these needs, administrators and policymakers subsequently can improve overall buy-in across the district, rather than only from specific groups.
Implications
Collectively, these findings have broader implications for school-based emergency preparedness practices for faculty, staff, and administrators. First, consistent with previous research (e.g., Embry-Martin, 2017; Rider, 2015), the introduction of the training component in conjunction with the lockdown drills is necessary to increase perceived preparedness. In other words, it is not sufficient to have people practice what to do—they must know why they are doing it so that they feel empowered to respond in times of crisis. To this end, it is important not only to conduct initial training but also to review and refresh on the concepts routinely. The school district for the current project, for example, has since incorporated a condensed version of the training provided by the research team into annual compliance training that all faculty, staff, administrators, and contract employees must complete before they are able to return to the school buildings each year. This serves to refresh them on the concepts they learned during the initial training in an effort to not only sustain but continue to grow their feelings of preparedness.
Second, the differences in perceptions based on respondent role highlight two important considerations for schools. First, for faculty, the findings suggest the need to incorporate additional emergency drills, such as lockouts, shelter-in place, or hold-in-place, to continue to increase familiarity with their response. As noted, the absence of significant differences across time for faculty participants related to lockdown and evacuation is likely a function of these drills already being something they practice; that familiarity likely fosters confidence in the response procedures. By incorporating other types of drills into the rotation, similar confidence may be achieved. Moreover, the findings highlight the importance of including all school employees in the emergency preparedness process. Crises in schools do not discriminate based on a person’s role; therefore, it is imperative that everyone have the necessary tools to keep themselves and others safe.
Limitations and Directions for Future Research
The present study is, of course, not without its limitations. First, the present study was conducted in an urban school district, despite that school shootings more commonly occur in suburban areas (Pane, 2018). Accordingly, future research should replicate the present study in areas of varying sizes and urbanicity to better understand how community-level factors impact emergency preparedness. Additionally, assessments of safety and preparedness in the present study were made in the context of lockdown drills that were performed. Future research may wish to make similar assessments following other types of emergency preparedness drills, such as lockouts or evacuations, to determine what, if any, impact the type of scenario practiced has on feelings of safety and preparedness. Although the study drew upon Protection Motivation Theory (Floyd et al., 2000), future research also should test this theory more explicitly by assessing educators’ vulnerability and perceptions of potential consequences, as well as response efficacy and self-efficacy.
Additionally, the surveys were self-report and in close proximity to the drills, so future research should employ other methods (e.g., behavioral observations) and long-term follow up. Further, since the district wanted all faculty and staff trained without a delay, we were unable to use a control group, which would have allowed for more causal interpretations about the effects. The response rate for the surveys also was relatively low but aligned with the average response rates reported meta-analyses about survey research (Cook et al., 2000). Future research studies should employ a randomized controlled design (using a wait-list control group for the training) and structure and incentivize participants for completing measures.
Finally, findings from supplementary analyses highlighted that race, ethnicity, and sex are important considerations for perceptions of school safety and emergency preparedness. Our sample was limited, however, in being predominantly White, female, and middle-aged. This reflects a larger issue that K-12 educators are primarily White, which does not reflect the racial and ethnic diversity of the students they serve (Billingsley et al., 2019). Furthermore, teachers from racial and ethnic minority backgrounds are more likely to work in urban, high poverty schools with poor working conditions and high turnover (Ingersoll et al., 2019). Research with more diverse samples of school staff should be conducted to also allow for the examination of intersecting effects of these demographics in perceptions of school safety and emergency preparedness, particularly in light of findings on the racial and ethnic disparities in relation to school emergency preparedness in both our study and previous research (Saberian et al., 2018; White et al., 2016). This research also could then differentiate race, ethnicity, and sex differences among respondents in various positions (e.g., faculty, staff, and administrators).
Conclusion
Despite these limitations, the present study makes an important contribution to the literature on lockdown and emergency preparedness drills specifically, as well as school safety more broadly. Although statistically rare events (Schildkraut et al., 2018), school shootings do happen in U.S. educational institutions each year, as do other types of emergencies for which preparedness plans are needed. Utilizing evidence-based approaches to responding to these crises is imperative to ensure safety of those within the school community. Continued practice and assessment are needed to not only ensure that, in the event of an emergency, everyone is ready to respond, but also to bolster buy-in and feelings of preparedness before the worst day arrives.
Footnotes
Acknowledgements
The authors wish to thank the editor and anonymous reviewers for their helpful critique and feedback on an earlier version of this manuscript.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This study was funded by the Syracuse City School District with assistance from the Hartford Insurance Group.
Notes
Author Biographies
Amanda B. Nickerson, Ph.D., NCSP is a professor of school psychology and director of the Alberti Center for Bullying Abuse Prevention at the University at Buffalo, the State University of New York. Her research focuses on school crisis prevention and intervention, violence and bullying prevention, and building social-emotional strengths of youth.
Kirsten R. Klingaman is a graduate of the Department of Criminal Justice at the State University of New York (SUNY) at Oswego. Her research interests include sociology of crime, mass shootings, and public policy.
