The Staff Perceptions of Behavior and Discipline Survey

Abstract

The practices of schoolwide positive behavior support (SWPBS) are dependent on staff implementation in classroom and common areas throughout the school. Thus, gaining the support and commitment of school staff is a critical step toward reaching full implementation of SWPBS. However, achieving buildingwide support can be challenging; many schools fail to reach full implementation due to staff resistance. The Staff Perceptions of Behavior and Discipline (SPBD) survey was developed to help teams assess staff beliefs about behavior and discipline and their perceptions of schoolwide expectations, school climate, systemic supports, and resources. With a better understanding of staff perceptions and beliefs, teams can proactively address staff needs and concerns and build on existing capacities. The current study describes the development and explores the psychometric properties of the SPBD in a sample of 1,210 school staff from 36 schools. The results of exploratory factor analysis provide evidence of internal validity, and the results of hierarchical linear modeling analyses examine concurrent validity with key variables such as level of SWPBS implementation, staff knowledge, and training.

Keywords

schoolwide positive behavior support systems change staff perception survey

Schoolwide positive behavior support (SWPBS) is an evidence-based approach for restructuring school discipline systems through a multitiered framework of social, emotional, and behavioral supports (Horner et al., 2009). By defining, teaching, and acknowledging schoolwide expectations, while also defining and applying consistent consequences for violations of school rules, all staff and students know what is expected across school settings. Although meaningful and positive outcomes are associated with SWPBS, including reduced rates of office disciplinary referrals, detentions, and suspensions (e.g., Bohanon et al., 2006; Bradshaw, Reinke, Brown, Bevans, & Leaf, 2008) and increased instructional time (e.g., Lassen, Steele, & Sailor, 2006; Luiselli, Putnam, Handler, & Feinberg, 2005), many schools struggle to reach and sustain full implementation of SWPBS.

Shifting from a traditional model of discipline to SWPBS requires a substantial change in the practices of staff, and obtaining full staff support and commitment to SWPBS can be a challenging endeavor. In the past decade, several studies have underscored the importance of staff support in all stages of implementation. In their interviews of team representatives from high and low implementing schools, Kincaid, Childs, Blase, and Wallace (2007) found that lack of staff support was the most frequently identified barrier to achieving full implementation. Team leaders reported that misunderstandings, philosophical beliefs incongruent to SWPBS, and limited knowledge of behavioral principles were all factors influencing the implementation of SWPBS in their schools. In a follow-up study, SWPBS facilitators identified major barriers of implementation at the universal level including teacher skepticism that SWPBS was needed, a belief that SWPBS was ineffective, and philosophical differences with the core elements of SWPBS, such as equating external positive reinforcement with bribery and overreliance on punishment (Lohrmann, Forman, Martin, & Palmieri, 2008). Team members supporting student needs at the individual level reported that teacher perceptions were the most pervasive barriers to implementation (Bambara, Nonnemacher, & Kern, 2009). These perceptions included philosophical beliefs inconsistent with SWPBS, limited knowledge of SWPBS principles, and difficulties in collaborative problem solving with other staff and families. In more recent research, staff support or buy-in has been identified as an influential factor to the sustainability of SWPBS (Coffey & Horner, 2012).

Although research on the impact of staff perceptions and support on SWPBS implementation is emerging, systemic change researchers have long asserted the importance of stakeholder support for an innovation and the importance of directly assessing their needs, concerns, and perceptions of a change initiative (e.g., Adelman & Taylor, 2001, 2007; Fullan, 2001; Fullan, Miles, & Taylor, 1980; Fuller, 1969; Hall & Hord, 2011; Newlove & Hall, 1976). Needs assessments are encouraged during planning, initial implementation, and continuously during all implementation phases. Gaining an understanding of staff perspectives and beliefs is a key component to a needs assessment. It can help researchers and practitioners anticipate staff needs and concerns, create more effective models of professional development, and mitigate barriers to achieving systemic change (Guskey, 1988; Hall & Hord, 2011; Nagle & Gagnon, 2008; Rogers, 2003).

Just like we in the field of SWPBS use student data to guide our support planning, it may be as important to gather data from school staff, the very people charged with implementation. Research suggests that 80% of staff support or “buy-into” SWPBS prior to implementation (e.g., Handler et al., 2007; Muscott et al., 2004; Sugai & Horner, 2006), but the SWPBS literature has not elucidated how teams assess this level of support (beyond a simple vote) and use this information to develop action plans. Although staff surveys exist in the field (e.g., the Effective Behavior Support Survey; Hagan-Burke et al., 2005), they serve to examine the status of SWPBS practices across levels of prevention and school settings. These surveys do not assess staff needs, concerns, beliefs, and attitudes as they pertain to SWPBS.

Researchers, teacher educators, administrators, behavior support coaches, and leadership team members may be better equipped to help staff shift to SWPBS if they were able to reach a deeper understanding of staff needs and their perceptions of behavior and discipline. To this end, this study introduces an instrument, the Staff Perceptions of Behavior and Discipline (SPBD). This survey was developed to help teams assess staff beliefs about behavior and discipline and their perceptions of schoolwide expectations, school climate, systemic supports, and resources. In this article, we outline the process of development, psychometric properties of the tool, and the relationship of the tool with other implementation factors.

Purpose

The purpose of this study was to develop and evaluate a tool designed to help teams understand the perceptions, needs, concerns, and existing capacities of the staff in their school. This study was comprised of three phases: (a) the development of the tool, (b) an exploration of the internal validity of the core items, and (c) an analysis of relationships between the core items and key school and staff variables, including level of SWPBS implementation, school level, staff knowledge, support, and training.

Phase 1: SPBD Survey Development

Literature Review

To develop the content of the SPBD, a team of faculty and graduate students conducted a review of the SWPBS and the systems change literature bases. Four search strategies were used to achieve a comprehensive review. First, we identified relevant literature through PsycInfo and ERIC databases using the following search terms: positive behavior supports, discipline reform, staff perceptions, school improvement, organizational change, system change, staff resistance, and implementation science. Second, we used the reference lists of relevant articles to secure literature we may have missed during our initial search of the computer databases. Third, we examined books in the fields of SWPBS and systemic change. Last, we explored the “research,” “publications,” and “resources” links within SWPBS and systemic change websites. We conducted all searches with the primary intent to identify staff-related factors that may act as barriers or facilitators to change. Both theoretical and empirical pieces were included in the review. We excluded pieces that were not published such as conference proceedings, dissertations, and position papers. The review of the literature revealed several common themes as described below. The SPBD items draw directly from these themes.

The connection between perceptions of the potential effectiveness of and need for an innovation and the level of implementation has been supported in the systemic change literature (e.g., Rogers, 2003) and in the SWPBS literature (e.g., Lohrmann et al., 2008). Members of a system are more motivated to change their daily practices when they perceive an impetus for change in the practices currently ingrained in the system (Adelman & Taylor, 2007; Noell & Gansle, 2009; Rogers, 2003). When staff perceive that there is a clear need for change in the discipline practices within their school system, they may be more likely to support implementation of SWPBS (Lohrmann et al., 2008; Sugai & Horner, 2006). However, it may not be sufficient for an approach to be perceived as effective when implemented in other schools. Staff must also perceive SWPBS as compatible with the school culture and the staff and students in their school (Lane et al., 2009; Lohrmann et al., 2008; Rogers, 2003).

When staff members believe the administration is committed to implementing a change initiative and will ensure necessary resources are provided to do so, staff are more likely to support and follow through with implementation. Likewise, when staff feel administration supports new practices and understands the risks associated with change, they may feel more secure and more apt to change their daily practices and take the necessary risks (Fixsen, Naoom, Blasé, Friedman, & Wallace, 2005; McGlinchey & Goodman, 2008). Hence, staff may be more likely to support implementation of SWPBS if they perceive that the school administrator is committed to SWPBS, actively involved in planning and implementation, and has the leadership skills to move implementation forward in a direction that will benefit the school community (Sugai, Horner, & McIntosh, 2008). However, if staff have experienced the frustration of observing many change initiatives come and go, and perhaps even administrators that have come and gone, they are more likely to perceive the initiative as ephemeral and simply wait for yet another fad to pass (Adelman & Taylor, 2007; Curtis, Castillo, & Cohen, 2008; Fullan, 2001; Hall & Hord, 2011; Kincaid et al., 2007; Lohrmann et al., 2008).

It is essential to secure supports and resources for implementation of an innovation, including materials, space, technology, time, and training (Adelman & Taylor, 2007; McGlinchey & Goodman, 2008; Rogers, 2003; Sugai et al., 2008). It is also important that staff are aware that these supports and resources exist and feel secure in knowing they will be provided what they need throughout all stages of SWPBS implementation (Bambara et al., 2009; Kincaid et al., 2007). Training and professional development is critical. If staff feel capable and confident in their ability to implement the necessary components of the innovation within their current job role, they may be more likely to support and follow through with implementation (Hall & Hord, 2011; Rogers, 2003; Tillery, Varjas, Meyers, & Collins, 2010). Therein, when staff feel confident in their ability to implement proactive behavior management strategies, they are more likely to do so (Morin, 2001).

In that systemic change efforts such as SWPBS require a culture of collaboration, a climate of mutual support, cohesiveness, and trust is essential. Change may be facilitated among members in a system when they perceive a sense of shared ownership and believe they are working together toward a common goal (Hall & Hord, 2011; McGlinchey & Goodman, 2008). Given that disciplinary challenges are frequently coupled with stress and other negative emotions, it may be of particular importance that staff receive ample opportunities to problem solve with colleagues in a safe, professional learning environment. If, however, staff feel disenfranchised, perceive resistance in their colleagues, or have few opportunities for collaboration, they may be less likely to embrace the practices of SWPBS (Kincaid et al., 2007; Lohrmann et al., 2008).

Last, the compatibility between the underlying philosophy of an innovation and the philosophy of staff may have a considerable impact on the adoption of an innovation (Adelman & Taylor, 2007; Fullan, 2001). Misperceptions, misunderstandings, and outright disagreement with the philosophy of SWPBS have been identified as barriers to implementation of SWPBS (Bambara et al., 2009; Kincaid et al., 2007; Lohrmann et al., 2008). Staff may believe that systems of extrinsic positive reinforcement are detrimental to students and rely on punishments as the primary response to behavioral violations. They may believe student behavior can only be changed by parents or guardians and there is very little that can be changed in the school setting. Staff may believe the practices of SWPBS are unfair because some students receive “special treatment.” The environmental modifications that address the antecedents to behavioral problems may be viewed as coddling the student or blaming staff for student behavior (Bambara et al., 2009; Kincaid et al., 2007; Lohrmann et al., 2008). When staff hold beliefs that are incongruent with the SWPBS philosophy, implementation may be hindered.

Content Validity Pilot

The SPBD items were developed to align with the content identified in the literature review above. The items of the SPBD were structured to limit response bias commonly associated with the assessment of beliefs and perceptions. The 24 Likert-type scale core items contain four response options: strongly disagree, disagree, agree, and strongly agree. A 4-point scale with no neutral answer option was chosen to alleviate ambivalence (e.g., Chang, 1994; Nowlis, Kahn, & Dhar, 2002). Also, items were worded to measure philosophies that can facilitate or create tensions in the implementation of SWPBS. For example, instead of creating a general, positively worded item to assess one’s beliefs about the use of rewards such as, “Students should be acknowledged for positive behavior,” we chose to word the item more specifically: “I believe we should reserve rewards for exceeding expectations, not simply for meeting them.” We theorized that the former item would be more susceptible to acquiescence and socially desirable responding, whereas the latter item better targets potential philosophical tension in the implementation of SWPBS. To attain a self-report of implementation integrity, a variable associated with staff perceptions and beliefs, three items ask respondents to rate the extent to which they currently implement the schoolwide expectations for teaching and acknowledging behavioral expectations and enforcement of consequences for behavioral violations.

In addition to the 24 core items of the SPBD, supplementary items were designed to gather additional information that is likely to be useful to planning teams. These additional items ask respondents to identify their job role, years of experience, and hours of professional development or training in SWPBS. Also, an item asks the respondent to self-assess his or her knowledge of SWPBS:

“When it comes to the concepts and procedures of SWPBS, my level of understanding is”:

Unfamiliar, I don’t know what it is.

Limited, I need to learn more.

Basic, I could implement.

High, I could teach others.

An item asks the respondent to rate their level of support for SWPBS:

“If you are familiar with SWPBS, please indicate your current level of support or commitment”:

I strongly disagree with this effort.

I disagree with this effort, but I will not resist it.

I agree with this effort, but I do not plan to participate in leadership or committee work.

I strongly agree with this effort; I plan to actively support it.

Last, qualitative or open-ended items were added to gather more specific information about staff concerns, needs, and existing capacities. These items are intended to provide more specific feedback to school-based planning teams and were not included in the present study. The open-ended statement of concern is used to assess staff concerns for SWPBS. This item, “When you think about [schoolwide positive behavior support], what concerns do you have? Please be frank and answer in complete sentences” (Hall & Hord, 2011, p. 79), is an established method of assessing stakeholder concerns for an innovation and is a component of the Concerns-Based Adoption Model (CBAM). The item “When it comes to behavior and discipline, what is working well in this school?” is used to assess existing capacities. To assess needs, the SPBD includes the item “What is needed to make it better?”

The SPBD was first piloted in 11 schools in the Western Washington region. Staff in 6 elementary schools, 3 middle schools, 1 high school, and 1 alternative middle and high school completed the survey. In addition to the data obtained by the school staff completing the survey in the 11 schools (n = 188), three administrators, six support personnel including SWPBS coaches and school psychologists, two university faculty with systemic change and SWPBS expertise, and six graduate students provided feedback on content of the survey by answering the following questions:

Do you feel this survey assesses the staff-related factors that can affect buy-in and implementation of SWPBS?

Are there items you would add or delete?

Do you feel this survey would help you plan and implement SWPBS in your school?

Also, they were asked to provide feedback on the wording:

Did you find any of the items confusing or difficult to understand?

If so, how might we improve the clarity or readability of the items?

Items were revised in accordance to the feedback we received. Also, three Likert-type scale items were added to address salient themes that emerged in responses to the open-ended items in the piloting phase. Specifically, because staff frequently commented on the need for students to be more responsible for their own behavior, for fellow staff members to implement SWPBS more consistently, and for administrators to get “tougher” on behavioral violations, items were added to the survey to assess these perceptions directly.

The SPBD is an online survey readers can access free of charge via a companion site developed for the survey (see www.spbdsupport.com) or by emailing the lead author. After a request is placed, a survey link is generated, and school officials send this link to certificated and classified staff who work directly with students. Responses to the survey are summarized using basic descriptive statistics in a report generated for each school.

Phase 2: Internal Validity

Exploratory factor analysis of the survey items was conducted after the survey was revised to determine how the survey items might best be grouped into subscales.

Setting and Participants

Schools

Participating schools included 36 schools from nine districts in Western Washington that were either implementing or preparing to implement SWPBS. Results of the Schoolwide Evaluation Tool (SET) administered for each school are provided in Table 1 (Sugai, Lewis-Palmer, Todd, & Homer, 2001). As not all schools were interested in collecting SET data, these data are available for 30 of the 36 schools. Districts included five urban, two suburban, and two rural districts with predominately White student populations. In most districts (Districts 1, 3, 4, 5, and 7), the second most predominate racial/ethnic group was Latino (21%, 7.6%, 22%, 24%, and 24%). In Districts 8 and 9, the next predominate group was Asian/Pacific Islander (26%, 18%). In District 2, the next predominate racial/ethnic group was African American (22%), and in District 6, the next predominate group was two or more races (11%). There were 25 elementary schools, 8 middle schools, and 3 high schools participating in this phase of the study. The schools had an average of 910 students enrolled and 56% of students on average received free or reduced price lunches. Schools were invited to participate in the study based on the interest of the school administration and/or leadership teams in developing an understanding of staff perceptions related to behavior, discipline, and SWPBS. See Table 1 for additional information on SET scores and response rates for schools and their districts.

Table 1.

Participating Schools, School Evaluation Tool (SET), and Staff Perceptions of Behavior and Discipline (SPBD) Survey.

School level	District	School code	SET total	Students enrolled	SPBD n	% Certified response^a	% F/RPL^b
Elementary	District 1	1	80	737	30	71	63
		2	71	716	29	66	21
		3	83	241	19	75	89
		4	80	741	27	92	64
		5	65	362	31	100	92
		6	96	526	31	100	62
		7	NA	483	29	78	83
		8	NA	641	22	71	16
	District 2	9	49	371	40	100	78
		10	68	362	23	95	92
		11	51	261	18	89	48
		12	51	502	25	78	86
		13	90	263	25	90	79
		14	58	423	26	93	81
		15	46	503	34	86	93
		16	44	444	31	96	52
		17	51	398	30	86	60
		18	45	416	29	88	83
		19	50	278	11	43	90
	District 3	20	100	527	38	100	26
		21	91	368	15	50	41
		22	86	487	20	74	38
		23	94	235	20	100	41
	District 6	24	NA	564	20	50	40
	District 7	25	NA	412	25	91	54
Middle	District 1	26	85	920	43	89	59
		27	39	784	36	55	32
		28	73	866	37	53	83
	District 2	29	64	770	22	41	62
	District 3	30	86	816	46	78	34
	District 4	31	88	736	58	90	74
	District 5	32	NA	786	57	89	49
	District 8	33	NA	1,229	64	81	43
High	District 3	34	63	1,217	49	52	26
	District 2	35	21	1,418	60	67	72
	District 9	36	39	1,613	90	93	40
Total			M = 61	M = 910	M = 34	M = 77	M = 56

Note. Participating schools, n = 36. Survey responses, N = 1,210. SET = School Evaluation Tool; SPBD = Staff Perceptions of Behavior and Discipline; F/RPL = free and reduced prices lunches; NA = not applicable.

The number of classified staff working with students could not be confirmed. Only certificated response rates are available. ^bPercentage of students receiving F/RPL.

Participants

A total of 1,210 SPBD survey responses were received and included in these analyses. Of the staff responding to the SPBD, 67.6% (n = 818) were certificated staff, 17.2% (n = 208) were classified staff, 8% (n = 97) were support staff, 3.2% (n = 39) were administrators, and the remaining 4% (n = 48) reported their role as “other.” Of the total responses, 55.4% (n = 670) were received from staff in 25 elementary schools, 28.2% (n = 341) were received from staff in 8 middle schools, and 16.4% (n = 199) were received from staff in 3 high schools. Response rates ranged from 41% to 100%, with higher response rates noted in the elementary schools (M = 82%) as compared with the secondary schools (M = 71%; see Table 2).

Table 2.

Five-Factor Solution and Psychometric Properties of the SPBD Scale (N = 1,210).

			Factor loadings
Item	M	SD	1	2	3	4	5
I don’t have time to teach the schoolwide behavioral expectations.	3.19	0.642	.722
Schoolwide behavior support may work in other schools, but I doubt it will work in ours.	3.33	0.588	.640
We should not have to teach students how to behave at school.	3.29	0.732	.613
I resent being asked to do one more thing in my classroom.	3.10	0.642	.602
I feel that rewarding students is the same as bribing them.	3.09	0.617	.554
I have trust in my administrator’s ability to lead us through change.	3.04	0.874		.776
The climate at this school is positive.	2.83	0.750		.750
Overall, I am satisfied with my job.	3.19	0.686		.696
I believe our school has (or will have) the necessary resources to support schoolwide positive behavior support.	2.83	0.721		.557
Schoolwide behavior supports are likely to be yet another fad that comes and goes in this school.	2.88	0.763		.379
Currently, I teach the agreed upon schoolwide behavior expectations to students.	2.63	0.904			.867
Currently, I acknowledge/reward students for meeting the agreed upon schoolwide behavior expectations.	2.82	1.01			.839
Currently, I apply the agreed upon schoolwide disciplinary consequences	2.71	1.12			.775
When problem behaviors occur, we need to get tougher.	2.41	0.733				.720
The students at this school need to be held more responsible for their own behavior.	1.81	0.709				.684
Parents in the community don’t seem to care about how their children behave at school.	2.75	0.697				.589
I believe we should reserve rewards for students exceeding expectations, not simply for meeting them.	2.57	0.822				.574
If students aren’t disciplined at home, they aren’t likely to accept any discipline at school.	2.68	0.818				.478
The staff at this school tends to resist change with concerns such as “we don’t do it that way here.”	2.84	0.704					.733
This school has successfully implemented change efforts in the past.	2.24	1.39					.778
My colleagues and I share a common philosophy for behavior and discipline.	2.39	1.10					.747
I suspect that my colleagues will not (or are not) consistently implementing the agreed upon schoolwide behavior plan.	2.56	0.757					.509
Behavior plans do not work well in our school.	2.65	1.18					.535
I am unaware of schoolwide changes that affect the staff and students.	3.03	0.871

Note. SPBD = Staff Perceptions of Behavior and Discipline.

Procedures

In the spring of 2013, the SPBD survey data were collected via an online survey distributed in an email sent to all staff regularly interacting with students in the participating schools. Both certificated and classified staff members were asked to complete the survey. To encourage frank responses, identifying information was not collected from respondents.

Results

Factor Analysis

The core 24 items on the SPBD were subjected to principal components analysis (PCA) using SPSS Version 21. Prior to performing the PCA, the suitability of data for factor analysis was assessed. Data with missing cases were excluded pairwise and 94% of cases had no missing data. Overall, the analysis revealed strong internal consistency for the SPBD, with an overall Cronbach’s alpha coefficient of .80. Inspection of the correlation matrix revealed the presence of many coefficients .3 and above. The Kaiser–Meyer–Oklin (KMO) value was .83, exceeding the recommended value of .6 (Kaiser, 1970, 1974) and Bartlett’s (1954) Test of Sphericity exceeded statistical significance, supporting the factorability of the correlation matrix.

PCA was performed utilizing an orthogonal varimax rotation and revealed the presence of five components with eigenvalues exceeding 1, explaining 19.9%, 10.6%, 7.7%, 6.2%, and 5.8% of the variance, respectively, and 54.6% cumulatively. One item (“I am unaware of schoolwide changes that affect the staff and students”) was not correlated at .3 or above with any of the five components and, therefore, the decision was made to eliminate the item from the final survey. An inspection of the screeplot revealed a clear break after the fifth component. Review of the five factors revealed a theoretical fit with the literature on SWPBS and systemic change. Therefore, it was decided to retain all five components for the final survey.

The five factors identified as part of the SPBD have moderate to strong coefficient alphas as follows:

Factor I. Teaching and acknowledging expectations: Effectiveness and need (.67)

Factor II. Systems: Resources, supports, and climate (.73)

Factor III. Implementation integrity (.79)

Factor IV. Philosophical views of behavior and discipline (.62)

Factor V. Systems: Cohesiveness and openness to change (.58)

The reader is referred to Table 2 for a complete list of the 24 core items within the five components and Table 3 for the psychometric properties of the survey components.

Table 3.

Psychometric Properties of the SPBD Scale (N = 1,210).

Factor	No. of items	n	M	SD	Skewness	Kurtosis	Cronbach’s α
Teaching and acknowledging behavioral expectations	5	1,206	15.99	2.13	−0.128	−0.120	.67
Sustainable change in school climate	5	1,204	14.78	2.64	−0.629	0.775	.73
Implementation integrity	3	1,147	8.15	2.55	−1.960	3.54	.79
Philosophical views of behavior and discipline	5	1,198	12.23	2.49	−0.273	0.123	.62
Cohesiveness and openness to change	5	1,207	12.69	3.25	−0.384	−0.103	.58
Total	23	1,131	64.00	8.32	−0.182	0.071	.80

Note. SPBD = Staff Perceptions of Behavior and Discipline.

Phase 3: Relationship to Key Variables

Multilevel modeling was used to explore the relationship of the SPBD to school- and staff-level variables.

Settings and Participants

Schools

This phase of the current study includes a subset of the data analyzed in Phase 2. Only the schools in which the level of SWPBS implementation was assessed through the SET were included in these analyses. This includes 30 schools from five school districts from Western Washington: 21 elementary schools, 6 middle schools, and 3 high schools. The participating schools included in this phase are identified in Table 1 with SET scores (n = 30).

Participants

A total of 993 survey responses were included in these analyses. This includes 67.5% (n = 670) teachers, 17% (n = 115) support personnel, 11.6% (n = 169) classified staff, and 4% administrators, and “other” (n = 39). Of the total responses, 57.8% (n = 574) were from 21 elementary schools, 22.1% (n = 220) from 6 middle schools, and 20% (n = 199) from 3 high schools.

Participants rated their knowledge, commitment or support, and training as related to SWPBS. Of the participants included in this phase, 4.2% (n = 42) reported that they were completely unfamiliar with SWPBS, 24.8% (n = 246) reported a partial or limited understanding, 52% (n = 516) reported a basic understanding, and 19% (n = 188) reported expert-level knowledge. Of the participants who were aware of SWPBS, the majority was supportive; 60.3% (n = 596) reported that they strongly supported SWPBS and planned to actively support it, and 23.4% (n = 231) reported that they supported the effort. Only 1.8% (n = 18) reported that they disagreed but did not intend to resist the effort and 2% (n = 20) reported that they strongly disagreed with SWPBS. Participants also reported the amount of SWPBS professional development they had received over the last 2 years: 30.6% (n = 304) reported that they received no professional development, 35.8% (n = 355) reported that they had received 1 to 3 hr of professional development, 19.6% (n = 195) reported 4 to 10 hr, 7.3% (n = 73) reported 11 to 20 hr, and 3.1% (n = 31) reported 20 or more hours of professional development.

Procedures

In the spring of 2013, SPBD data were gathered from certificated and classified staff regularly interacting with students. Within 1 to 3 weeks after the SPBD was sent to staff, the level of SWPBS implementation was assessed in each participating school by the SET. The SET was administered by the research team and standardized procedures as detailed in the SET manual were followed. The SET along with the administration and scoring manual are readily available on the positive behavior supports technical assistance center website (see www.pbis.org).

The SET is an established measure designed to evaluate the level of SWPBS implementation by assessing the critical components of SWPBS including Expectations Defined, Behavior Expectations Taught, Ongoing System for Rewarding Behavior Expectations, System for Responding to Behavioral Violations, Monitoring and Decision-Making, Management, and District-Level Support. The SET has robust test–retest reliability (M = 97.3), interobserver agreement (M = 99%), internal consistency (α = .96), construct validity (Pearson’s r = .75), and sensitivity to change (Horner et al., 2004). The SET procedures comprise a review of permanent products, observations of the school common areas, hallways, and classrooms, brief interviews with students and staff, and an extended interview with the school administrator. The summary score, the average percentage of implementation across SET domains, is a general index of SWPBS implementation. To reach an adequate level of implementation, schools must reach or exceed the criterion of 80% on the total SET and 80% on the domain of Behavioral Expectations Taught.

Data Analyses

Multilevel modeling was used to take into account the clustering of the participants within schools and to determine the relationship of the SPBD to variables commonly associated with SPBD. These variables include SWPBS implementation level (total SET score), school level (i.e., elementary, middle, high), level of understanding of SWPBS, level of support for SWPBS, and hours of training. Models were fitted using STATA software (StataCorp, 2013).

Results

SWPBS Implementation

As stated above, the SET was used to assess the level of SWPBS implementation in the participating schools. The elementary school SET scores ranged from 44% to 100%. Of the 21 elementary schools, 9 reached or exceeded the criterion for full implementation. The SET scores from the secondary schools ranged from 21% to 88%. Of the 9 secondary schools, 3 schools scored above the 80% criterion. Table 1 provides the SET scores for each school.

Hierarchical Linear Modeling (HLM) Analysis

Table 4 summarizes the results of the three-level model and depicts estimate changes as the variables were added. Each added variable significantly improved the model fit. In Model 1, the SPBD was significantly related to the SET (coefficient = 0.0043; p < .001). Model 2 revealed a significant relationship between the SPBD and SET scores (coefficient = 0.0031; p < .01) and school level (coefficient = −0.1124; p < .01). Participant variables related to SWPBS were added in Model 3, including level of understanding of SWPBS, level of support for SWPBS, and hours of SWPBS professional development. When these variables were added to the model, a significant relationship between the SPBD and the SET was no longer found. Instead, we found significant relationships among school level (coefficient = −0.1027; p < .01), knowledge or understanding of SWPBS (coefficient = 0.0531; p < .01), commitment or support of SWPBS (coefficient = 0.0499; p < .01), and hours of training (coefficient = 0.0169; p < .01). Whereas a direct relationship was found between the scores on the SPBD and staff support, hours of training, and level of understanding, an inverse relationship was found between SPBD and school level. As staff support for SWPBS, number of hours of training, and level of understanding increased, SPBD scores also increased. However, as level of school increased, SPBD scores decreased.

Table 4.

HLM Analyses of SPBD, School-Level, and Respondent-Level Variables (n = 993).

	Model 1	Model 2	Model 3
Parameter	Coefficient (SE)	Coefficient (SE)	Coefficient (SE)
Intercept	2.5356 (0.09)***	2.7746 (0.10)***	2.4951 (0.10)***
SET scores	0.0043 (0.00)**	0.0031 (0.00)**	0.0018 (0.00)
School level		−0.1124 (0.03)**	−0.1027 (0.03)**
Level of understanding or knowledge			0.0531 (0.02)**
Support or commitment			0.0499 (0.01)***
Professional development			0.0169 (0.01)**
Random effects
Level 1 variance	0.1094532	0.1095201	0.0999621
Level 2 variance	0.0161788	0.0105083	0.010069
Chi-square	11.77	27.67	124.75
Likelihood ratio test		9.40	92.87

Note. Chi-square value significant at p < .001 level. HLM = hierarchical linear modeling; SPBD = Staff Perceptions of Behavior and Discipline; SET = Schoolwide Evaluation Tool.

**p < .01. ***p < .001.

Discussion

The purpose of this study was to present the SPBD, a survey tool designed to help SWPBS teams better understand the unique insights, concerns, needs, and strengths of the staff in their school. Although this understanding is an important component to achieving full and sustained implementation of SWPBS, an instrument of this kind did not previously exist. The SPBD shows promise as a tool to assist teams in making data-driven decisions for staff supports as they plan for and implement SWPBS.

The three phases of the current study provide evidence for the technical properties of the SPBD. In the first phase, we selected the content of the SPBD through a review of the systems change and SWPBS literature. The items were structured to include several formats, providing opportunities for teams to collect both quantitative and qualitative data. The SPBD was piloted, and items were revised. In the second phase, the internal validity of the SPBD was explored through PCA of the core items. In addition to overall strong reliability for the SPBD, the analyses revealed five distinct and reliable components of the SPBD that may be used to effectively summarize and interpret survey responses: (a) Teaching and Acknowledging Expectations; (b) Systems: Resources, Supports, and Climate; (c) Implementation Integrity; (d) Philosophical Views of Behavior and Discipline; and (e) Systems: Cohesiveness and Openness to Change. The scales identified in the SPBD align to the themes identified in the SWPBS and systemic change literature.

In the final phase, HLM was used to examine relationships to key school- and staff-level variables that can influence staff perceptions of student behavior and disciplinary practices. The results provide evidence of the concurrent validity of the SPBD and provide insight into relationships among staff perceptions of SWPBS and level of SWPBS implementation in the school, amount of training received, and understanding of SWPBS. The results revealed that staff in implementing schools with more training and better understanding of SWPBS tended to report the most positive perceptions of behavior and discipline. As suggested in previous research, as staff better understand positive behavioral practices, they may have more favorable perceptions of these practices, and they are more apt to implement them (e.g., Morin & Battalio, 2004). Our results also revealed an inverse relationship between staff perceptions of SWPBS and the level of the school; as school level increased, staff scores on the SPBD decreased. This finding has implications for teams, and indicates that staff in elementary schools may have more positive perceptions of SWPBS than staff in secondary schools. This finding is also supported by previous research that indicates more staff in secondary schools may have perceptions of behavior and discipline that can cause tensions in SWPBS implementation, such as reliance on punishment (Flannery, Frank, Kato, Doren, & Fenning, 2013).

However, the design of this study only allows for the examination of associations among variables and does not allow for the identification of causal relationships. More research is needed to examine the exact nature of the relationships among level of implementation, school level, staff support for SWPBS, and staff knowledge and training. Although response rates in the current study were high for certificated staff, it should be noted that response rates could not be verified for classified staff who comprised 17% of the participants. Hence, the application of these findings to classified staff is limited.

The SPBD is designed to assist school-based leadership teams as they plan for or continue with implementation of SWPBS. Administered during the pre-implementation planning phase, the SPBD can help teams identify needs and existing capacities critical to readiness. Administered annually, the SPBD may help teams identify and proactively address emerging needs and concerns and thereby create a continuous cycle of improvement critical to sustainability. Examination of results from the SPBD in this manner can help teams anticipate staff needs and tailor training or models of professional development to target staff needs. Furthermore, interpretation of the SPBD results at the domain and item levels can be enlightening for teams. In our experiences working with schools, we have helped teams use data from the Philosophical Views of Behavior and Discipline domain to target professional development and structure topics for dialogue within professional learning communities. Moreover, staff responses to qualitative items offer rich insights about their needs and concerns and the existing capacities of the school.

Conclusion

The importance of staff perceptions to the successful implementation of SWPBS is increasingly evident in research conducted over the last decade. However, no tool has been available to help teams reach an understanding of staff perceptions, so they may proactively design supports to address staff needs and concerns. Although the SPBD shows promise to serve this need, further exploration of the psychometric properties of the SPBD and its relationship to other variables relevant to the implementation of SWPBS, such as school climate and student-outcome variables, is warranted. There is also need for further study of how staff perceptions and beliefs change over time with the implementation of SWPBS. In systemic change, it is important to be mindful that the identification of evidence-based practice is just one part of the process. To fully bridge the research to practice gap and help bring SWPBS to scale, it may be just as important to understand the concerns, needs, and insights of the very people we are asking to change.

Footnotes

Action Editor: Lee Kern

Authors’ Note

The views expressed in this article are the personal views of the author and do not purport to represent the official views of the Office of Disability Employment Policy or the U.S. Department of Labor.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Adelman

H. S.

Taylor

(2001). Organization facilitators: A change agent for systemic school and community changes. Los Angeles, CA: Center for Mental Health in Schools.

Adelman

H. S.

Taylor

(2007). Systemic change for school improvement. Journal of Educational and Psychological Consultation, 17, 55–77.

Bambara

Nonnemacher

Kern

(2009). Sustaining school-based individualized positive behavior support: Perceived barriers and enablers. Journal of Positive Behavior Interventions, 11, 161–176.

Bartlett

M. S.

(1954). A note on the multiplying factors for various chi square approximations. Journal of the Royal Statistical Society, Series B, 16, 296–298.

Bohanon

Fenning

Carney

Minnis

Anderson-Harriss

Moroz

. . . Piggott

(2006). School-wide application of positive behavior support in an urban high school: A case study. Journal of Positive Behavior Interventions, 8, 131–145.

Bradshaw

Reinke

Brown

Bevans

Leaf

(2008). Implementation of school-wide positive behavioral interventions and supports (PBIS) in elementary schools: Observations from a randomized trial. Education & Treatment of Children, 31, 1–26.

Chang

(1994). A psychometric evaluation of 4-point and 6-point Likert-type scales in relation to reliability and validity. Applied Psychological Measurement, 18, 205–215.

Coffey

Horner

(2012). The sustainability of schoolwide positive behavior interventions and supports. Exceptional Children, 78, 407–422.

Curtis

M. J.

Castillo

J. M.

Cohen

R. M.

(2008). Best practices in systems-level change. In Thomas

Grimes

J. P.

(Eds.), Best practices in school psychology V (pp. 887–902). Bethesda, MD: National Association of School Psychologists.

10.

Fixsen

D. L.

Naoom

S. F.

Blasé

K. A.

Friedman

R. M.

Wallace

(2005). Implementation research: A synthesis of the literature (FMHI Publication No. 231). Tampa: University of South Florida, Louise de la Parte Florida Mental Health Institute, National Implementation Research Network.

11.

Flannery

K. B.

Frank

J. L.

Kato

M. M.

Doren

Fenning

(2013). Implementing schoolwide positive behavior support in high school settings: Analysis of eight high schools. The High School Journal, 96, 267–282.

12.

Fullan

(2001). The “new” meaning of educational change (3rd ed.). New York, NY: Teachers College Press.

13.

Fullan

Miles

Taylor

(1980). Organization development in schools: The state of the art. Review of Educational Research, 50, 121–183.

14.

Fuller

F. F.

(1969). Concerns of teachers: A developmental conceptualization. American Educational Research Journal, 6, 207–226.

15.

Guskey

T. R.

(1988). Teacher efficacy, self-concept, and attitudes toward the implementation of instructional innovation. Teaching and Teacher Education, 4, 63–69.

16.

Hagan-Burke

Burke

Martin

Boon

Fore

Kirkendoll

(2005). The internal consistency of the Effective Behavior Support Survey. Education & Treatment of Children, 28, 400–413.

17.

Hall

G. E.

Hord

S. M.

(2011). Implementing change: Patterns, principles, and potholes (3rd ed.). Boston, MA: Allyn & Bacon.

18.

Handler

M. W.

Rey

Connell

Their

Feinberg

Putnam

(2007). Practical considerations in creating school-wide positive behavior support in schools. Psychology in the Schools, 44, 29–39.

19.

Horner

R. H.

Sugai

Smolkowski

Eber

Nakasato

Todd

A. W.

Esperanza

(2009). A randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. Journal of Positive Behavior Interventions, 11, 133–144.

20.

Horner

R. H.

Todd

A. W.

Lewis-Palmer

Irvin

L. K.

Sugai

Boland

J. B.

(2004). The school-wide evaluation tool (SET): A research instrument for assessing school-wide positive behavior support. Journal of Positive Behavior Interventions, 6, 3–12.

21.

Kaiser

(1970). A second generation Little Jiffy. Psychometrika, 35, 401–415.

22.

Kaiser

(1974). An index of factorial simplicity. Psychometrika, 39, 31–36.

23.

Kincaid

Childs

Blase

Wallace

(2007). Identifying barriers and facilitators in implementing school-wide positive behavior support. Journal of Positive Behavior Interventions, 9, 174–184.

24.

Lane

K. L.

Kalberg

J. R.

Bruhn

A. L.

Driscoll

S. A.

Wehby

J. H.

Elliott

(2009). Assessing social validity of school-wide positive behavior support plans: Evidence for the reliability and structure of the Primary Intervention Rating Scale. School Psychology Review, 38, 135–144.

25.

Lassen

Steele

Sailor

(2006). The relationship of school-wide positive behavior support to academic achievement in an urban middle school. Psychology in Schools, 43, 701–712.

26.

Lohrmann

Forman

Martin

Palmieri

(2008). Understanding school personnel’s resistance to adopting school-wide positive behavior support at a universal level of intervention. Journal of Positive Behavior Interventions, 10, 256–269.

27.

Luiselli

J. K.

Putnam

R. F.

Handler

M. W.

Feinberg

A. B.

(2005). Whole-school positive behaviour support: Effects on student discipline problems and academic performance. Educational Psychology, 25, 183–198.

28.

McGlinchey

M. T.

Goodman

(2008). Best practices in implementing school reform. In Thomas

Grimes

(Eds.), Best practices in school psychology V (pp. 853–874). Bethesda, MD: National Association of School Psychologists.

29.

Morin

(2001). Winning over the resistant teacher. Journal of Positive Behavior Interventions, 3, 62–64.

30.

Morin

Battalio

(2004). Constructing misbehavior: The efficacy connection in responding to misbehavior. Journal of Positive Behavior Interventions, 6, 251–254.

31.

Muscott

H. S.

Mann

Benjamin

T. B.

Gately

Bell

K. E.

Muscott

A. J.

(2004). Positive behavior interventions and supports in New Hampshire: Preliminary results of a statewide system for implementing school-wide discipline practices. Education & Treatment of Children, 27, 453–457.

32.

Nagle

Gagnon

(2008). Best practices in planning and conducting needs assessments. In Thomas

Grimes

(Eds.), Best practices in school psychology V (pp. 2207–2223). Bethesda, MD: National Association of School Psychologists.

33.

Newlove

B. W.

Hall

G. E.

(1976). A manual for assessing open-ended statements of concern about an innovation. Retrieved from ERIC database. (ED144207)

34.

Noell

G. H.

Gansle

K. A.

(2009). Moving from good ideas in educational systems change to sustainable program implementation: Coming to terms with some of the realities. Psychology in the Schools, 46, 79–89.

35.

Nowlis

Kahn

Dhar

(2002). Coping with ambivalence: The effect of removing a neutral option on consumer attitude and preference judgments. Journal of Consumer Research, 29, 319–334.

36.

Rogers

E. M.

(2003). Diffusion of innovations (5th ed.). New York, NY: Free Press.

37.

StataCorp. (2013). Stata Statistical Software: Release 13. College Station, TX: Author.

38.

Sugai

Horner

(2006). A promising approach for expanding and sustaining school-wide positive behavior support. School Psychology Review, 35, 245–259.

39.

Sugai

Horner

McIntosh

(2008). Best practices in developing a broad-scale system of school-wide positive behavior support. In Thomas

Grimes

J. P.

(Eds.), Best practices in school psychology V (pp. 765–780). Bethesda, MD: National Association of School Psychologists.

40.

Sugai

Lewis-Palmer

Todd

A. W.

Homer

R. H.

(2001). School-wide evaluation tool (Version 2.0). Eugene: University of Oregon.

41.

Tillery

A. D.

Varjas

Meyers

Collins

A. S.

(2010). General education teachers’ perceptions of behavior management and intervention strategies. Journal of Positive Behavior Interventions, 12, 86–102.