Does It Matter Who Evaluates Teachers? Principal Versus Teacher-Led Evaluation and Teacher Motivation

Abstract

Increasing job demands and continuing struggles to improve teacher evaluation practice raise the question of how peers might assist principals with teacher evaluation. Using a robust international sample (TALIS2013) of 36,411 teachers from 2,759 schools in 11 countries, we tested the hypothesis that teacher-led evaluation practices are associated with more teacher-reported positive changes in classroom practice, confidence, and motivation than principal-led evaluation practices in three areas evaluation: (1) classroom observations, (2) assessments of teacher content knowledge, and (3) analysis of student test score data. We found that teacher-led evaluation is associated with more positive feelings of motivation and change in practice for all three evaluation areas, but particularly for assessments of teacher content knowledge and test score data analysis. Further, principals’ reported use of extrinsic motivational tools to reward or punish teachers based upon their evaluation was also negatively associated with teachers’ motivation and reports of positive change in practice.

Keywords

teacher evaluation teacher motivation instructional improvement supervision principal observation school leadership

Introduction

In their role as supervisors, principals have traditionally conducted teacher evaluations. During Race to the Top (RttT), which had an explicit goal of increasing the effectiveness of teacher evaluations, principals continued to shoulder the primary responsibility of formative and summative teacher evaluations, including: pre- and post-observation conferences, conducting formal and informal observations, providing feedback to teachers, and writing up summative evaluations. In some cases, such demands required that principals spend even more time observing teachers and documenting their effectiveness (Donaldson & Woulfin, 2018; Lavigne & Good, 2015). According to principals, accommodating these time demands has been an enormous challenge (Donaldson & Woulfin, 2018; Goldring et al., 2015; Kraft & Gilmour, 2016; Lavigne & Chamberlain, 2017). To cope, principals have resorted to cutting observations short, double-dipping, or reducing the number of observations they conduct (Donaldson & Woulfin, 2018; Stecher et al., 2018), calling into question the quality and/or utility of such feedback (Kraft & Christian, 2022). Even with the increased flexibility in teacher evaluations, principals have experienced little change in these instructional leadership demands since the passing of the Every Student Succeeds Act. In addition to the demands, the costs to schools and districts are substantial. It is estimated that it costs close to $700 million for principals to evaluate all 3.1 million U.S. K-12 public school teachers twice in a given year (Dynarski, 2016). Given these substantial investments, understanding if and how the information generated from teacher evaluation systems is translating into improved teaching and learning is critical.

The principalship has become, in recent years, increasingly challenging and complex work (Lavigne & Good, 2019; Ridge & Lavigne, 2020). With all of the pressures placed on principals, teacher supervision and evaluation frequently take a back seat. This is exacerbated by the increased evaluation loads and documentation demands from recent reform efforts that have resulted in principals spending more time writing their evaluations and recording data from observations instead of observing teachers and providing teachers with rich feedback (Flores & Derrington, 2017; Kraft & Christian, 2022). Furthermore, principals struggle to provide feedback that is content-based as they often lack critical pedagogical knowledge but often observe teachers in content areas outside of their expertise (Donaldson & Firestone, 2021; Kraft & Gilmour, 2016). As such, almost half of teachers report that the feedback they receive from their principal is not useful (Cherasaro et al., 2016). Together these findings point to many barriers to using principal-led teacher evaluations to improve teaching and learning. This is coupled with evidence that new teacher evaluation reform efforts supported by principal-led evaluations have generally failed to improve teaching and learning, particularly in the United States (Kim & Sun, 2021; Lavigne & Good, 2019).

This raises the question of how the underutilized practice of peer-based evaluation approaches (e.g., coaching and/or peer assistance and review; Goldstein, 2007; Harvey et al., 2019; Johnson, 2019; Johnson et al., 2010; Ridge & Lavigne, 2020) might be useful in supporting (or replacing) principal-led teacher evaluations as the primary means by which teachers are evaluated, thus potentially reducing the burden currently shouldered by administrators (Galey-Horn & Woulfin, 2021). Although peer-based evaluation approaches offer substantial promise and are commonly practiced outside of the U.S., it has continued to be a relatively uncommon practice in the U.S. (Johnson, 2019). As such, these approaches have been understudied and largely absent from reviews of effective teacher evaluation systems in the U.S. (National Council on Teacher Quality, 2018).

However, programs that utilize expert or more experienced peers as evaluators have some demonstrable benefits, according to the literature (Johnson, 2019; Johnson et al., 2010; Papay & Johnson, 2012). While not without their own set of organizational and financial challenges, there is some evidence that peer-coaching/evaluation models might reduce teacher turnover as well as improve teacher quality, instruction, and with it, teacher confidence (Johnson, 2019; Papay & Johnson, 2012). Likewise, teacher-led observation provides a mechanism to help schools and districts consider different possibilities for separating teacher supervision and evaluation. Oftentimes the principal is the coach that supervises teachers throughout the school year, providing formative feedback, and also serves as the judge in the summative task of teacher evaluation. This structure results in supervision traveling under the guise of evaluation (Glanz & Hazi, 2019; Hazi, 1994), which is a problem that continues to plague teacher evaluation even under the Every Student Succeeds Act. Twenty-five states continue to embed formative feedback within the summative portion of their teacher evaluation models (Mette et al., 2020).

Yet a transition away from administrator-led evaluation toward more peer-led models should be based upon evidence that teachers and their practice would benefit from the change, and, in this case, more evidence is needed. The literature does seem clear that teachers are more likely to improve in their craft when they have experienced evaluators that deeply understand teaching and know what teachers need/want to learn (Donaldson & Firestone, 2021; Ford, 2018; Kraft & Christian, 2022). An important part of understanding the potential benefits of teacher-led evaluation concerns teacher cognitive sensemaking about their evaluation experience and how it does (or does not) motivate change in practice (Donaldson, 2021; Ford et al., 2017, 2018). There is growing evidence to suggest that fellow teachers have strong potential to be effective evaluators of other teachers’ practice, particularly when the evaluation emphasizes formative approaches (Donaldson, 2021; Johnson, 2019). There are more obvious reasons why this might be the case: experienced teachers are more likely to have intimate knowledge of practice to help propel novice teachers forward and they can be tapped based upon needed content area expertise (Johnson, 2019). Yet there is also evidence that how evaluation is framed including how teachers experience the evaluation process can influence teachers’ perceptions of the credibility, utility, and/or validity of the feedback and thus their likelihood of using it for instructional improvement (Cherasaro et al., 2016; Ford, 2018; Ford et al., 2017; Jiang et al., 2015; Pallas, 2021; Reddy et al., 2018). There is arguably no one in the school building better equipped to understand how teachers experience evaluation and thus no one more likely to prioritize teachers’ needs throughout the evaluation process than a fellow teacher. However, direct comparisons of the outcomes of administrator versus teacher-led evaluations are largely absent within the literature.

Therefore, the purpose of this paper was to test the hypothesis that teacher-led evaluation practices are related to more teacher-reported positive changes in classroom practice, increased confidence, and motivation than principal-led evaluation practices. To test this hypothesis, we used a robust international sample of schools from the TALIS 2013 to acquire the kind of variation in teacher evaluation practice needed to draw meaningful conclusions. Evidence supporting our hypothesis would be seen in the form of a positive relationship between teacher/mentor-led evaluation and teacher positive change in motivation/practice and/or a negative relationship between principal/management-led evaluation and teacher positive change in motivation/practice. The three research questions below which guided the study derived from a central, overarching question: Does who is doing the evaluation matter?

In their review of effective teacher evaluation systems, the National Council on Teacher Quality (2018) outlines seven principles that were common across all models: multiple measures of teacher effectiveness, student surveys, objective measures of student growth, at least three rating systems, annual evaluations and observations of all teachers, professional development that is tied to teacher evaluation, and written feedback after each observation. Recognizing this, in this study, we focus on the most critical as well as common and enduring components of teacher evaluation models: the use of observational data—also one of the most commonly used components in TALIS-participating sites (Organization of Economic CoOperation and Development [OECD], 2013)—and analysis of test score data (Lavigne & Good, 2020). However, with the flexibility of the Every Student Succeeds Act in mind, we also consider more fine-grained measures and practices such as the use of teacher content knowledge assessments. Thus our specific research questions each address one of the formal evaluation components we have chosen:

Is there a relationship between who is leading the formal observations of classroom teaching and teacher perceived positive change in motivation/practice?

Is there a relationship between who is leading the assessment of teacher content knowledge and teacher perceived positive change in motivation/practice?

Is there a relationship between who is leading the analysis of student test score data and teacher perceived positive change in motivation/practice?

Teacher Evaluation: Then and Now

Glanz and Hazi (2019) trace the development of teacher evaluation back to the origins of supervision, a tool that emerged to provide efficiency for superintendents during the rapid economic growth that transformed schools during and following the industrial revolution (Tyack, 1974). At that time, supervision was established as a role that fell under the purview of school administration (Payne, 1875). In the 1900s, scientific management and efficiency pushed for supervision as a means of coordinating and controlling labor (Bobbitt, 1913; Taylor, 1911). According to Glanz and Hazi, the 1920s marked a shift toward a more democratic and middle-manager as opposed to a bureaucratic approach under the responsibility of school administration (Dewey, 1929; Hosic, 1920; Newlon, 1923). Supervision began to establish its professional autonomy and then merged with the field of curriculum. In the 1990s, the accountability movement elevated teacher evaluation and supervision began to function incognito. This shift coupled with the emergence of instructional leadership meant that teacher supervision and evaluation primarily resided under the role of school leaders (Glanz & Hazi, 2019).

During the last decade, reform efforts have placed teacher evaluation at the center of accountability and school improvement efforts and have focused on the effectiveness of principals in that role. Launched in 2009, Race to the Top allocated funds to states to develop comprehensive systems of improvement in four areas: college and career readiness, teaching and leadership, data systems and technology to improve instruction, and turnaround efforts for low-performing schools. Reforms to teacher evaluation fell under teaching and leadership and included the use of standardized tests as a significant measure of teaching effectiveness, multiple rating categories, and high-stakes teacher evaluations (Lavigne & Good, 2019). While the 19 states that won Race to the Top dollars were significantly more likely to implement Race to the Top policies and offer less flexibility (Gagnon et al., 2017), its effects on teacher evaluation were widespread with changes noted in even 56% of the states that never applied for Race to the Top (Howell, 2015).

Today, many of the components of teacher evaluation systems that emerged in Race to the Top continue to sustain under the Every Student Succeeds Act (ESSA) despite greater flexibility (Lavigne & Good, 2019; Walsh et al., 2017). One important change under ESSA is that student achievement is no longer required as a significant measure in a teacher evaluation model (U.S. Department of Education, 2015). This has been the area that has changed the most as states have made modifications. In particular, states have decreased their use of value-added models (VAMs) within their teacher evaluation systems and have made steps to elevate and emphasize formative feedback practices, reversing some of the changes to teacher evaluation models from the Race to the Top era (Close et al., 2020).

Principal-Led Evaluations

One aspect of teacher evaluations that has remained relatively constant over the years is the primary role of the principal. It also seems clear that, since RttT, teachers are receiving, on average, more feedback than in times past (Donaldson, 2021). However, the documentation demands of recent reform efforts have been so burdensome that principals spend more time writing their evaluations and recording data from observations than they do observing teachers and providing them with rich feedback (Flores & Derrington, 2017). This has also hindered principals’ ability to provide frequent feedback (Kraft & Gilmour, 2016), document deficiencies (Range et al., 2011), and rate teachers ineffective (Kraft & Gilmour, 2017). Principals have cut observations short, double-dipped or reduced the number of observations, or have been unavailable to address teacher concerns (Donaldson & Woulfin, 2018; Stecher et al., 2018).

Given the overwhelming emphasis on training principals on new teacher evaluation policies, models, and observation instruments, principals still receive limited training on how to provide feedback despite plentiful science on cognition and performance appraisal to inform such training (see Donaldson, 2021 for a review). As such, principal evaluations tend toward positive affirmations of teachers’ practices rather than suggestions for improvement (Kraft & Gilmour, 2017). This is compounded by the fact that principals oftentimes observe and provide feedback to teachers in content areas outside of their area of expertise (Cherasaro et al., 2016) which results in principals providing feedback almost entirely focused on pedagogy (Kraft & Gilmour, 2016). Likewise, principals and districts often have different responses and agency in relation to teacher evaluation that prompts them to tinker, modify, and adapt teacher evaluation systems to align with their beliefs and to meet local needs and concerns (J. Cohen et al., 2020; Donaldson & Woulfin, 2018; Marsh et al., 2017; Woulfin et al., 2016). To a certain degree, these behaviors belie a recognition, however implicit, that there are aspects of evaluation policy and practice in need of adjustment or reconsideration.

While, in one study, teachers indicated that they received timely, frequent, and accurate feedback that included specific suggestions for improvement, just a little more than half (55%) felt that the feedback they received was useful. Further, only 62% indicated they had access to professional development, 61% had planning time to implement new strategies, and 60% had support from an instructional leader. Notably, only 33% had access to observing an expert teacher model feedback (Cherasaro et al., 2016). Other studies have found similar teacher perceptions and rates of support for evaluation (see Firestone & Donaldson, 2019 for a review). Teachers who do not receive specific feedback tend to report lower self-efficacy (E. C. Smith et al., 2020); furthermore, failing to address content knowledge translates to missed opportunities to increase teacher effectiveness (Hill et al., 2008). Together these findings point to many barriers to using principal-led teacher evaluations to improve teaching and learning and indicate a general paucity of opportunities to learn from their peers in meaningful ways, including through feedback on instruction.

The Potential of Teacher-Led Evaluation and Enhancing Motivation for Improvement

Fortunately, greater flexibility under the Every Student Succeeds Act offers the opportunity to think about teacher evaluation in different ways, something that has not yet been realized in most states (Kim & Sun, 2021; Mette et al., 2020). Some have advocated for a more facilitative rather than evaluative role of the school leader that focuses primarily on supporting and developing teachers in the refinement of their skills and practices (Ford, 2018; Ford & Hewitt, 2020; Guenther, 2021; Murphy et al., 2013). The inherent challenges of principal-led evaluation raise the possibility of considering the underutilized practice of peer coaching and observation (Harvey et al., 2019; Ridge & Lavigne, 2020), of which there is growing empirical support in the literature (Donaldson, 2021; Galey-Horn & Woulfin, 2021; Johnson, 2019; Johnson et al., 2010; Papay & Johnson, 2012).

In their analysis of TALIS 2013 data, Ford et al. (2018) found that teachers who were primarily evaluated by a fellow teacher as opposed to the principal reported higher overall job satisfaction. Motivation theory, while underutilized in understanding the effects of evaluation approaches and tools on teacher motivation (Donaldson, 2021; Ford & Hewitt, 2020; Ford et al., 2018), provides some explanation as to why peer-based evaluation models might make for happier teachers, for generating better feedback, and ultimately inducing teachers to use that feedback to improve their practice. There is ample evidence to suggest that teachers, on the whole, are attracted to and remain in the profession for primarily intrinsic reasons (Kennedy, 2005; Lortie, 1975; Watt & Richardson, 2008). Having entered willingly into a human improvement enterprise (D. K. Cohen, 2011), teachers are naturally enthusiastic about their profession, invested in seeing the children under their care succeed, and are typically willing to use any and all means to make that happen—including working to improve their teaching (Kunter & Holzburger, 2014; Roth, 2014). Teachers, in general, do not object to being evaluated—in fact, many request to be—they object to the way it is often carried out (Ford et al., 2017; Guenther, 2021; Hewitt, 2015; Johnson, 2019). This is partly because many evaluation systems (in the US in particular), continue to misconstrue lack of improvement and/or poor performance as a lack of motivation on the part of teachers and thus continue to heavily rely on extrinsic motivational mechanisms such as rewards (pay-for-performance) and punishment (cut pay, censure, termination, and job/grade changes; Ford et al., 2018). While using reward and punishment might work for individuals who are less-than-enthusiastic about their work, activating existing intrinsic motivation requires a very different approach—one which is focused on ensuring that conditions are ripe for individuals’ innate orientations to their work, learning, and development to emerge (i.e., teachers’ psychological needs; Ryan & Deci, 2017). Extensive research has found that extrinsic motivational mechanisms are likely to: (a) only change behavior in the short-term, (b) damage intrinsic motivation and its long-term development, and (c) induce gaming practices that result from reward or punishment being attached to outcomes instead of behaviors (Ryan & Brown, 2005; Ryan & Deci, 2017; Ryan & Weinstein, 2009).

Addressing teachers’ psychological needs for learning and development is something that teacher/peer-led evaluation seems uniquely suited to do. In studies of teacher evaluation, teachers often desire and request more and better feedback on their practice (Donaldson, 2021; Ford, 2018; Gabriel & Woulfin, 2017; Johnson, 2019). Providing constructive feedback along with support to address the areas of their practice that need improvement is key to building a teachers’ feelings of competence and self-efficacy (Ford et al., 2017; Lavigne & Good, 2015; E. C. Smith et al., 2020). Assuming that teacher-led evaluations foster more content-specific feedback—an important feedback characteristic that principals struggle to provide (Kraft & Gilmour, 2016; Smith et al., 2020)—teaching peers are more likely to have a grasp on subject-specific concerns (Kraft & Gilmour, 2016). Moreover, more experienced teachers, with their wealth of knowledge of practice, have a greater capacity to show teachers where they are missing the mark and model alternative approaches (Johnson, 2019). They are also able to help them set realistic goals for student learning growth and provide a structure, process, and point person (or people) for reviewing progress toward meeting those goals (Longchamp, 2017). Perceived expertise of the evaluator in an of itself enhances the credibility of the feedback and, with it, its likelihood of being used by the evaluee to improve their practice (Donaldson, 2021; Ford, 2018). Second, with models such as peer assistance and review (PAR), observations are more frequent and intentional, yielding the perception that the evaluation process as a whole is meaningful and that the feedback is more representative of their “true” classroom practice (Ford, 2018; Johnson, 2019).

The need for autonomy is also critical for teachers throughout the evaluation process. Having voice, choice, and a semblance of control in how evaluation is conducted as well as the goal setting, assessments, and data used to evaluate is important for meeting this need (Donaldson, 2021; Lavigne & Good, 2015). Having peers lead evaluations can disrupt hierarchical power dynamics between the principal and teacher in traditional administration-led evaluations, increasing the potential for collaboration and engagement so it feels less like evaluation is “done to them” (Danielson, 2010). Furthermore, teacher emotions play an important role in how teachers respond to evaluation and so having some consideration of teachers’ fears and concerns when engaging in evaluation can help them overcome the initial negative responses they often have to receiving feedback. The power dynamic which invariably exists between principals and teachers can make this difficult to do and it can also limit the degree to which teachers perceive the principal as a colleague offering constructive advice instead of a supervisor providing a summative performance evaluation (Donaldson, 2021). Because of the highly personal nature of feedback, teaching peers who share in the experience of teaching and evaluation with all of its inherent challenges and rewards are more likely to develop a relationship that mitigates the typical affective and motivational barriers to change and improvement that arise in evaluation situations (Donaldson, 2021; Lavigne & Good, 2020; Ryan & Deci, 2017). This is not to say that principals could not learn how to fulfill these roles, but there is much evidence to suggest that they are, on the whole, not yet provided the opportunity and structures to do so.

Method

Sample

The primary data source for this study was the 2013 Teaching and Learning International Survey (TALIS 2013), administered by the Organisation for Economic Co-operation and Development (OECD). The data analyzed in this study were collected from all teachers who work in lower secondary schools (level 2 of International Standard Classification of Education [ISCED]) from a selected number of schools and nations out of the 32 included in the TALIS 2013 dataset and international report (OECD, 2014b). While there is a more recent administration of the TALIS (TALIS 2018), TALIS 2013, unlike TALIS 2018, focused heavily on the appraisal of teachers’ work in schools and thus included the detailed information on these processes necessary for the analysis.

For each country sampled, TALIS 2013 set a target size of 200 schools with 20 teachers per school. Schools were selected according to a national sampling plan which used systematic random sampling with probability proportional to size (PPS) within explicit strata which might include school types, regions, or funding (OECD, 2014a). In selecting a sample of TALIS countries appropriate to answer our research questions, we adopted a purposive sampling approach to maximize variability in our focal variable, teacher or leader-led evaluation. Thus, we utilized an empirical approach to country selection, including both countries where a target proportion of the schools in the country reported engaging in either a high proportion of teacher or leader-led evaluation (a high proportion of teacher-led evaluation being a relative criterion as fewer schools and countries engage in this as a primary form of evaluation). This approach was informed by global perspectives on accountability contexts (see Holloway et al., 2017). Using vignettes from South Korea, the United States, Hungary, and Mexico, W. C. Smith (2014) argues that testing for accountability purposes is an increasingly common perspective being adopted globally. W. C. Smith and Kubacka (2017) argue that while rarely used in isolation, the use of student test scores to evaluate teachers is the most common approach and the extent to which test scores are emphasized impacts teachers’ perceptions of feedback effectiveness. Further, having school leaders or other administrators be responsible for leading the evaluation of teachers is part-and-parcel of the “managerial order” characteristic of accountability systems (Holloway et al., 2017).

Thus, our empirical approach was to separately examine all three analytic samples (i.e., teaching observations, assessments of content knowledge, and analysis of test scores) and determine a contrasting set of countries with very little (or no) teacher-led evaluation happening (i.e., primarily or exclusively principal/admin-led) or a relatively high proportion of engagement in teacher-led evaluation. Three countries that consistently reported strong leader-led teacher evaluation across the three analytic samples were the United States, England, and Chile and this appears consistent with what we know of these countries’ emphasis on accountability at the time the data were collected (Falabella & De La Vega, 2016).

For countries to be considered “high teacher-led evaluation” countries, a criterion for inclusion was established separately for each analytic sample relative to the frequency of teacher-led evaluation occurring in that feedback type. For the observation and content knowledge analyses, countries were included if more than 10% of the sample within a country was teacher-led. For the test score analysis, countries were included if they had greater than the average (arithmetic mean) number of teacher-led evaluation schools. Applying these criteria resulted in Brazil, France, Portugal being included as “high teacher-led evaluation” countries for the observation analysis (thus six countries total including the U.S., England, and Chile); Brazil, France, Portugal, Japan, Korea, Mexico, and Romania being included for the content knowledge analysis (thus 10 total countries); and Brazil, Japan, Korea, and Croatia for the test score analysis (thus seven total countries). The base analytic sample from which the sample for each of the three focal analyses was derived was 36,411 teachers within 2,759 schools in 11 countries.

Measures

Items from both the teacher and principal questionnaires were used in this analysis. Descriptive statistics for the pooled sample of 11 countries on all of the variables used in the regression analyses and their corresponding survey numbers in the TALIS surveys are provided in Table 1. Below we discuss specifically the outcome and key independent variable in the analysis.

Table 1.

Descriptive Statistics for Study Variables From the International TALIS 2013 Dataset.

	Original N	Imputed N	Min	Max	Mean	Imputed mean	SD	TALIS variable(s)
Teacher characteristics
Female	36,410	36,411	0	1	0.63	0.63	0.48	TT2G01
Teacher age	36,375	36,411	18.00	76.00	41.32	41.31	10.73	TT2G02
Full time teacher	35,561	36,411	0	1	0.78	0.78	0.40	TT2G03
Teacher has permanent contract	35,755	36,411	0	1	0.75	0.76	0.42	TT2G06
Proportion SPED students in class (most or all)	36,120	36,411	0	1	0.06	0.06	0.23	TT2G09
Participated in an induction program	36,138	36,411	0	1	0.57	0.57	0.49	TT2G19A
Teacher assigned a mentor	35,791	36,411	0	1	0.18	0.19	0.39	TT2G20A
High barriers to PD in last year	36,152	36,411	1	20.00	11.52	11.52	3.17	TT2G27C-G (α = .71)
Teacher job satisfaction	36,411	36,411	4.25	15.18	12.16	12.16	1.92	TT2G46A-J (α = .83)
Teacher collaboration intensity	36,301	36,411	1	6.00	3.31	3.31	0.98	TT2G33A-H (α = .76)
Tch pcpt. of shared responsibility and decision making	36,299	36,411	1	4.00	2.72	2.72	0.58	TT2G44A-E (α = .85)
Student learning climate	36,334	36,411	1	4.00	3.24	3.24	0.47	TT2G45A-D (α = .78)
Teacher feelings of self-efficacy	36,302	36,411	3.77	15.47	12.48	12.48	2.00	TSELEFFS
Teacher motivation and change in practice	36,411	36,411	−5.61	5.96	0.79	0.80	2.84	TT2G30F,H,J,L,N (α = .87)
Principal/School characteristics
Principal female	2,745	2,759	0	1	0.53	0.55	0.49	TC2G01
Village, <3,000 people	2,727	2,759	0	1	0.20	0.19	0.39	TC2G09
Small town, 3,001–15,000	2,727	2,759	0	1	0.21	0.23	0.42	TC2G09
Town, 15,001–100,000	2,727	2,759	0	1	0.24	0.25	0.43	TC2G09
City, 100,001–1,000,000	2,727	2,759	0	1	0.21	0.20	0.40	TC2G09
Large city, >1,000,000	2,727	2,759	0	1	0.13	0.12	0.33	TC2G09
Private school	2,746	2,759	0	1	0.22	0.21	0.41	TC2G10
Student/teacher ratio	2,599	2,759	0	558	16.83	16.67	18.19	TC2G12A/G14
High poverty school (more than 60%)	2,702	2,759	0	1	0.17	0.16	0.36	TC2G15C
Principal perception of climate	2,718	2,759	1	4	3.15	3.14	0.41	TC2G30A-F (α = .80)
Principal instructional leadership	2,718	2,759	1	4	2.93	2.93	0.55	TC2G21B-E (α = .77)
Principal job satisfaction	2,719	2,759	3.24	15.81	13.07	13.08	1.91	PJOBSATS
Principal use of extrinsic motivators following eval.	2,470	2,759	0.86	4.00	1.52	1.53	0.43	TC2G29C, E-H (α = .70)

Note. Raw values for all variables in this table, except the outcome, teacher motivation, and change in practice, in which Rasch scale scores are reported. Appropriate weights applied for descriptive statistics and reliability analyses.

TALIS 2013 utilized confirmatory factor analysis (a classical test theory [CTT] approach) for its measure construction and scaling (OECD, 2014a). The Rasch model, in contrast, is an Item Response Theory (IRT) approach and is distinguished from classical test theory in considering the ability of respondents in tandem with the difficulty of the items to which they are responding. Advantages and disadvantages of both notwithstanding (see, e.g., Singh, 2004), both approaches are useful in the development and scaling of latent measures; in fact, other prominent international education datasets have opted to use an IRT approach for measure construction and scaling (e.g., TIMSS and PISA). Our choice to adopt an IRT approach over CFA was due to the exploratory nature of our development of the teacher change in motivation/practice outcome measure, which required a wider range of information to assess person and item performance than is typically provided using a CTT approach.

In addition to a host of other diagnostic information, the WINSTEPS program produces a scaled-score for each teacher in log-odds units which represents where each teacher’s perceptions locate him/her on the continuum of teacher change in practice/motivation (low, negative values reflect perceptions of a less motivation and change in practice, and high, positive numbers reflect perceptions of more motivation and change in practice). We set our threshold at mean-squared values of 0.5 to 1.5—accepted thresholds for Winsteps analysis (Linacre, 2014). Items from both the teacher and principal questionnaires were used in this analysis.

Positive Change in Teacher Motivation and Practice

This was our primary outcome measure, which was constructed from teacher questionnaire items TT2G30F, H, J, L, M, and N. These items elicited teachers’ perception of the degree to which the feedback they received from their evaluation had resulted in a positive change in their: confidence, motivation, satisfaction, and teacher classroom practices including classroom management strategies and use of student assessments for learning. The item stem was: Concerning the feedback you have received at this school, to what extent has it directly led to a positive change in any of the following? (TT2G30). This variable was Rasch analyzed and the items and results of the Rasch analysis are located in Appendix A.

Leader/Teacher-Led Evaluation Practices

This variable derived from principal questionnaire items TC2G28A1-6, C1-6, and D1-6 which ask who in the school performs the following tasks as a part of the formal teacher evaluation process: (a) direct observation of classroom teaching, (b) assessments of teachers’ content knowledge, and (c) analysis of students’ test scores. For the purposes of this analysis, we dichotomized these items which classified schools as sites where evaluation was conducted by: (a) principal or school management personnel, (b) teacher or teacher mentors, and (c) for schools where both of the aforementioned groups conducted the evaluation. Schools that indicated no use of the practice or that it was conducted by external individuals or bodies were excluded entirely from the analysis.

Principal Use of Extrinsic Motivational Tools After Evaluation

One important factor in the effects on teacher motivation and practice following an evaluation is how principals attach rewards or consequences to the results (Ford, 2018). This composite measure includes items that ask principals about the frequency with which they tie extrinsic rewards or punishments to the teacher’s evaluation performance such as: changes to pay (either up, e.g., the use of bonuses, or down, e.g., censure or dismissal, or changes to job responsibilities). These were on a 4-point rating scale from “never” to “always.”

Analytic Strategy

Teachers in the international sample who were missing scores on any one of the items comprising the outcome were removed before analysis (i.e., the values were not imputed). Teacher change in motivation/practice, as well as several other similar perception items toward the end of the TALIS teacher survey (such as self-efficacy and climate perceptions) exhibited a unit nonresponse pattern (Enders, 2010). After reviewing the missing data coding procedure in the TALIS technical manual, the missing codes in the dataset indicated that nearly all of the teachers who were missing scores for the outcome either returned the survey blank or incomplete (OECD, 2014a).

In this case, we endeavored to determine whether or not there were significant differences between teachers who were missing a score for our outcome and other measured variables—in other words, could the data be assumed to be missing completely at random (MCAR), which is needed to justify listwise deletion. To test this assumption, we conducted a series of Bonferroni corrected t-tests and chi-squared tests of independence between teachers who had a complete score for our outcome and those for whom it was missing. We found no significant differences between the groups with respect to these covariates, and thus list-wise deletion was a justifiable missing data handling approach (Enders, 2010). After list-wise deletion was completed, missing data at the teacher level in study covariates was around 3%, while missing data at the school level was around 5%, and both exhibited a general item and unit non-response patterns (Enders, 2010). Then, we employed multiple imputation (MI) techniques to the raw, teacher and school-level data. Multiple imputation is substantially more robust than typical list or pair-wise deletion procedures to missing data bias, and results in multiple versions of the same dataset with different plausible values for the missing data based on available variable data and their underlying covariance structure (Enders, 2010).

Once teachers without scores on the teacher motivation/practice were removed, the remainder of the missing data exhibited a general item non-response pattern (De Leeuw et al., 2003). A general item non-response pattern manifests as gaps in item response that appear to be randomly dispersed throughout the dataset (i.e., MCAR). Because multiple imputation does not require us to invoke an assumption of MCAR for the results to be unbiased, and there was no evidence to suggest they were missing not at random (MNAR), we chose the less stringent assumption of MAR, which allows for missing data on a variable to be related to other measured variables in the analysis (Enders, 2010).

Two-Level HLM Analysis

Because there are typically several different dimensions to a teachers’ evaluation (e.g., lesson observations, test score growth, etc.), we did not assume that all aspects of the evaluation process exhibited differences depending on whether they were conducted by an administrator versus a fellow teacher; rather we investigated each practice separately with respect to this hypothesis. We used variable information from the principal questionnaire on teacher/leader-led evaluation practices (described above) to conduct three separate 2-level HLM analyses for each of our focal areas of teacher evaluation: (1) classroom observations, (2) assessments of teacher content knowledge, and (3) analysis of student test scores. Separate Level 2 files were created for each evaluation type with the same set of imputed level 1 files. A series of “do if” statements were run to flag school principals who indicated that they or members of school management conducted the formal evaluation procedure in question (principal/management), other teachers or assigned mentors (teacher/mentor) conducted the evaluation, or both. Any schools that indicated that the evaluation type in question was not used or done by external individuals or bodies were excluded from the analysis. This procedure led to an effective sample size for the classroom observation analysis of 16,428 teachers in 1,400 schools; teacher content knowledge analysis, 18,615 teachers within 1,441 schools; and for the test score analysis, 21,246 teachers within 1,667 schools.

In addition to the focal dummy predictors of primary evaluator (principal/management, teacher/mentor, or both), we included other important teacher and school characteristics, attitudes, and perceptions about school working conditions and teacher appraisal practices that were presumed to be related to the outcome based on prior research. School-level variables related to the outcome such as: school climate, principal, and school characteristics such as school climate, poverty, urbanicity, sector, and principal satisfaction as well as the primary evaluator dummy variables were used to model between-school variation in teacher change in practice/motivation. The teacher weight, TCHWGT, was incorporated into the final analysis to maintain the intended representativeness of the sample. All other teacher-level effects remained fixed at Level 2, as an analysis revealed that there was little between-school variation in the relationships between each of the teacher-level predictors and positive change in practice/motivation.

Finally, because the sample sizes in this analysis were relatively large and therefore substantial power to detect small effects was present, Keith’s (2015) effect sizes were used to provide additional context for findings. Standardized coefficients of β ≥ .05 reflected a small effect, β ≥ .10 reflected a medium effect, and β ≥ .25 reflected large effects. School-level sample sizes were sufficient to use robust standard errors to adjust for heavier tails present at the highest intervals of many of the climate measures in the TALIS dataset. Robust standard errors can reasonably adjust for any non-normality in the outcome that can lead to heteroscedasticity and thus provide more accurate significance tests and confidence intervals (Hox, 2010).

Findings

The first research question addressed differences in teacher perceptions of positive change in their motivation/practice with respect to who was responsible in the school building for conducting direct observations of their classroom practice. Table 2 displays these results. The null model ICC demonstrates that approximately 16% of the variation in teacher perception of positive change in practice/motivation was between schools. Model 1 presents only the main effects; Model 2 the main effects with teacher covariates; The final model includes all variables including school-level covariates.

Table 2.

HLM Analysis of Teacher Motivation and Change in Practice From Evaluation by Direct Observation of Classroom Teaching.

	Null		Model 1		Model 2		Final model
	β	SE ^a	β	SE ^a	β	SE ^a	β	SE ^a
Teacher fixed effects
Intercept $γ_{00}$	−.158	0.028***	.106	0.064	.076	0.071	.053	0.095
Female $γ_{10}$					.046	0.034	.043	0.034
Full time $γ_{20}$					−.098	0.045*	−.093	0.042*
Teacher age $γ_{30}$					−.084	0.020***	−.087	0.020***
Permanent contract $γ_{40}$					−.041	0.039	−.036	0.040
Prop. of SPED students in class (most or all) $γ_{50}$					−.067	0.074	−.074	0.074
Participated in an induction program $γ_{60}$					.044	0.039	.046	0.039
Has been assigned a mentor $γ_{70}$					.196	0.037***	.196	0.036***
High barriers to PD in last year $γ_{80}$					−.111	0.019***	−.111	0.019***
Teacher job satisfaction $γ_{90}$					.217	0.018***	.220	0.018***
Teacher collaboration intensity $γ_{100}$					.127	0.020***	.127	0.020***
Student learning climate $γ_{110}$					−.023	0.017	−.021	0.017
Shared responsibility and decision making $γ_{120}$					.200	0.017***	.199	0.017***
Teacher self-efficacy $γ_{130}$					.141	0.018***	.141	0.018***
School fixed effects
Evaluation by principal only $γ_{01}$			−.273	0.073***	−.232	0.062***	−.202	0.061***
Evaluation by both teachers and principal $γ_{02}$			−.313	0.079***	−.293	0.066***	−.226	0.063***
Female principal $γ_{03}$							.164	0.042***
Private school $γ_{04}$							.012	0.066
High poverty school $γ_{05}$							.053	0.051
Village, <3,000 people $γ_{06}$							−.051	0.077
Small town, 3,001–15,000 $γ_{07}$							−.094	0.071
Town, 15,001–100,000 $γ_{08}$							−.155	0.067*
City, 100,000–1,000,000 $γ_{09}$							−.174	0.079*
Student-teacher ratio $γ_{010}$							.108	0.034***
Principal instructional leadership $γ_{011}$							.034	0.024
Principal perception of climate $γ_{012}$							.005	0.020
Principal job satisfaction $γ_{013}$							−.040	0.021
Principal use of extrinsic tools after eval $γ_{014}$							−.050	0.019**
Random effects								Pseudo-r²
Level 1 between teachers	.991***		.990***		.728***		.728***	.26
Level 2 between schools	.192***		.186***		.140***		.119***	.38

Robust standard errors reported. Coefficient estimates in this table are the pooled estimates from the five imputed datasets provided by the HLM program, with the teacher weight applied (TCHWGT). All continuous variables standardized. Large cities (1,000,000+) and teachers primarily evaluated by other teachers/mentors were the comparison/holdout groups for those dummy variable sets.

***

p ≤ .001. **p ≤ .01. *p ≤ .05.

As the final model of evaluation by direct observation of teaching demonstrates, holding other covariates constant, teachers whose principal/administration were the sole observer(s) of classroom practice in their building reported significantly lower overall positive change in practice/motivation, β = −.202, SE = 0.061, p < .001, as compared to those who were observed by a fellow teacher or assigned mentor only, β = .053, SE = 0.095, p = n.s. In schools where both principals/administration and teachers were responsible for classroom observations, the results were very similar to those for principals only—nearly a quarter of a standard deviation lower on average—both of which are medium-to-strong effects, β = −.226, SE = 0.063, p < .001.

Covariates exhibiting medium-to-large positive effects on positive change in teacher practice/motivation were: teacher job satisfaction, β = .220, SE = 0.018, p < .001, shared responsibility and decision making, β = .199, SE = 0.017, p < .001, having been assigned a mentor, β = .196, SE = 0.036, p < .001. Positive medium-sized effects were found in teacher self-efficacy, β = .141, SE = 0.018, p < .001, collaboration intensity, β = .127, SE = 0.020, p < .001, and teaching in a school with a female principal, β = .164, SE = 0.042, p < .001. Teacher age, β = −.087, SE = 0.020, p < .001, and experiencing high barriers to professional development in the past year, β = −.111, SE = 0.019, p < .001, were small-to-medium-sized negative effects related to positive change in teacher practice/motivation. One final small negative relationship key to the investigation was found between positive change in teacher practice/motivation and principal’s use of extrinsic motivational tools after the observation of classroom teaching, β = −.050, SE = 0.019, p < .01.

The second research question addressed differences in teacher perceptions of positive change in their motivation/practice with respect to who was responsible in the school building for conducting assessments of teacher content knowledge. Table 3 displays these results. The null model ICC demonstrates that 20% of the variation in teacher perception of positive change in practice/motivation was between schools.

Table 3.

HLM Analysis of Teacher Motivation and Change in Practice From Evaluation Using Assessments of Teacher Content Knowledge.

	Null		Model 1		Model 2		Final model
	β	SE ^a	β	SE ^a	β	SE ^a	β	SE ^a
Teacher fixed effects
Intercept $γ_{00}$	−.017	0.032	.210	0.050***	.118	0.066	.270	0.089**
Female $γ_{10}$					.041	0.028	.042	0.029
Full time $γ_{20}$					−.038	0.042	−.036	0.039
Teacher age $γ_{30}$					−.076	0.020***	−.078	0.020***
Permanent contract $γ_{40}$					−.021	0.036	−.020	0.037
Prop. of SPED students in class (most or all) $γ_{50}$					−.076	0.078	−.076	0.078
Participated in an induction program $γ_{60}$					.069	0.040	.071	0.039
Has been assigned a mentor $γ_{70}$					.162	0.033***	.164	0.033***
High barriers to PD in last year $γ_{80}$					−.102	0.018***	−.104	0.019***
Teacher job satisfaction $γ_{90}$					.216	0.016***	.217	0.016***
Teacher collaboration intensity $γ_{100}$					.149	0.019***	.148	0.019***
Student learning climate $γ_{110}$					−.021	0.014	−.021	0.014
Shared responsibility and decision making $γ_{120}$					.189	0.018***	.188	0.018***
Teacher self-efficacy $γ_{130}$					.124	0.016***	.131	0.016***
School fixed effects
Evaluation by principal/admin. only $γ_{01}$			−.239	0.065***	−.224	0.060***	−.183	0.057***
Evaluation by both teachers and principal $γ_{02}$			−.302	0.076***	−.314	0.075***	−.252	0.065***
Female principal $γ_{03}$							.075	0.050
Private school $γ_{04}$							−.060	0.077
High poverty school $γ_{05}$							−.022	0.066
Village, <3,000 people $γ_{06}$							−.171	0.086*
Small town, 3,001–15,000 $γ_{07}$							−.271	0.081***
Town, 15,001–100,000 $γ_{08}$							−.235	0.063***
City, 100,000–1,000,000 $γ_{09}$							−.208	0.081**
Student-teacher ratio $γ_{010}$							.048	0.020*
Principal instructional leadership $γ_{011}$							−.057	0.029
Principal perception of climate $γ_{012}$							.040	0.022
Principal job satisfaction $γ_{013}$							−.029	0.021
Principal use of extrinsic tools after eval $γ_{014}$							−.090	0.021***
Random effects								Pseudo-r²
Level 1 within schools	.923***		.923***		.678***		.678***	.27
Level 2 between schools	.233***		.225***		.201***		.173***	.26

***

p ≤ .001. **p ≤ .01. *p ≤ .05.

As the final model of evaluation by using assessments of teacher content knowledge demonstrates, holding other covariates constant, teachers whose principal/administration were the sole evaluator of teacher content knowledge reported significantly lower overall positive change in practice/motivation, β = −.183, SE = 0.057, p < .001. This as compared to those who were observed by a fellow teacher or assigned mentor only, which was positively associated with teacher positive change in practice/motivation, β = .270, SE = 0.089, p < .01. In schools where both principals/administration and teachers were responsible for classroom observations, the results were very similar to those for principals only—one quarter of a standard deviation lower on average, β = −.252, SE = 0.065, p < .001. Thus, these results approach a nearly one-half standard deviation net difference in the positive change in practice/motivation between teachers evaluated by principals only versus teachers only (0.183 + 0.270 = 0.453).

Covariates exhibiting medium-to-large positive effects on positive change in teacher practice/motivation were similar to in the other models: teacher job satisfaction, β = .217, SE = 0.016, p < .001, shared responsibility and decision making, β = .188, SE = 0.018, p < .001, having been assigned a mentor, β = .164, SE = 0.033, p < .001. Positive medium-sized effects were found in teacher self-efficacy, β = .131, SE = 0.016, p < .001, and collaboration intensity, β = .148, SE = 0.019, p < .001. Experiencing high barriers to professional development in the past year, was a medium-sized negative effect related to positive change in teacher practice/motivation, β = −.104, SE = 0.019, p < .001. Emerging in this model again was a key negative association between positive change in teacher practice/motivation and principal’s use of extrinsic motivational tools after evaluation, β = −.090, SE = 0.021, p < .001.

The final research question addressed differences in teacher perceptions of positive change in their motivation/practice with respect to who in the school building is conducting evaluation using the analysis of student test scores. Table 4 displays these results. The null model ICC demonstrates that approximately 17% of the variation in teacher perceptions of positive change in practice/motivation was between schools.

Table 4.

HLM Analysis of Teacher Motivation and Change in Practice From Evaluation Using Analysis of Students’ Test Scores.

	Null		Model 1		Model 2		Final model
	β	SE ^a	β	SE ^a	β	SE ^a	β	SE ^a
Teacher fixed effects
Intercept $γ_{00}$	−.087	0.027***	.086	0.094	.043	0.125	.135	0.107
Female $γ_{10}$					.026	0.031	.026	0.031
Full time $γ_{20}$					−.031	0.044	−.036	0.042
Teacher age $γ_{30}$					−.076	0.017***	−.078	0.017***
Permanent contract $γ_{40}$					−.015	0.037	−.012	0.037
Prop. of SPED students in class (most or all) $γ_{50}$					−.048	0.064	−.050	0.064
Participated in an induction program $γ_{60}$					.101	0.035**	.100	0.035**
Has been assigned a mentor $γ_{70}$					.161	0.027***	.163	0.028***
High barriers to PD in last year $γ_{80}$					−.085	0.017***	−.088	0.017***
Teacher job satisfaction $γ_{90}$					.210	0.016***	.210	0.016***
Teacher collaboration intensity $γ_{100}$					.136	0.018***	.134	0.018***
Student learning climate $γ_{110}$					−.010	0.016	−.009	0.016
Shared responsibility and decision making $γ_{120}$					.203	0.017***	.202	0.017***
Teacher self-efficacy $γ_{130}$					.104	0.014***	.111	0.014***
School fixed effects
Evaluation by principal only $γ_{01}$			−.152	0.103	−.248	0.104*	−.199	0.096*
Evaluation by both teachers and principal $γ_{02}$			−.220	0.101*	−.314	0.102**	−.252	0.092**
Female principal $γ_{03}$							.101	0.047*
Private school $γ_{04}$							−.067	0.072
High poverty school $γ_{05}$							−.033	0.055
Village, <3,000 people $γ_{06}$							−.140	0.076
Small town, 3,001–15,000 $γ_{07}$							−.182	0.078*
Town, 15,001–100,000 $γ_{08}$							−.184	0.062**
City, 100,000–1,000,000 $γ_{09}$							−.162	0.073*
Student-teacher ratio $γ_{010}$							.036	0.013**
Principal instructional leadership $γ_{011}$							−.037	0.027
Principal perception of climate $γ_{012}$							.026	0.023
Principal job satisfaction $γ_{013}$							−.082	0.024***
Principal use of extrinsic tools after eval $γ_{014}$							−.082	0.020***
Random effects								Pseudo-r²
Level 1 within schools	.912***		.912***		.666***		.666***	.27
Level 2 between schools	.192***		.187***		.187***		.154***	.20

***

p ≤ .001. **p ≤ .01. *p ≤ .05.

Holding other covariates constant, teachers whose principal/administration were the sole evaluators involved in analysis of student test score data reported significantly lower overall positive change in practice/motivation, β = −.199, SE = 0.059, p < .05, as compared to those who were observed by a fellow teacher or assigned mentor only, β = .135, SE = 0.107, p = n.s. In schools where both principals/administration and teachers were responsible for classroom observations, the results were very similar to those for principals only—over one-quarter of a standard deviation lower on average, β = −.252, SE = 0.092, p < .01. Both the principal-only and principal-and-teacher evaluation effects were medium-to-strong in size.

Covariates exhibiting medium-to-large positive effects on positive change in teacher practice/motivation were similar to the other two models: teacher job satisfaction, β = .210, SE = 0.016, p < .001, shared responsibility and decision making, β = .202, SE = 0.017, p < .001, having been assigned a mentor, β = .163, SE = 0.028, p < .001. Positive medium-sized effects were found in teacher self-efficacy, β = .111, SE = 0.014, p < .001, collaboration intensity, β = .134, SE = 0.018, p < .001. Principal job satisfaction, β = −.082, SE = 0.024, p < .001, and experiencing high barriers to professional development in the past year, β = −.088, SE = 0.017, p < .001, were medium-sized negative effects related to positive change in teacher practice/motivation. Emerging in this final model, similar to the other two models, a small, negative association between positive change in teacher practice/motivation and principal’s use of extrinsic motivational tools after evaluation, β = −.082, SE = 0.020, p < .001.

In all three models discussed above, some notable small effects were those between teacher age and female principal. All three models clearly show that as teacher age increased, there was a corresponding small decrease of positive change in motivation/practice, β = −.078 to −.087 across the three models all p < .001. With the exception of the assessment of teacher content knowledge analysis, teachers in schools where the principal was female also had a small overall positive effect on teacher perceptions of positive change in motivation/practice, β = .164 and .101 for principal observations and analysis of test scores respectively.

One important consideration given study findings is how teachers might have differed in quality and experiences between schools where teachers were the primary evaluators versus other types of schools. It was certainly possible that schools where teachers were those leading evaluation were unique in some fashion: either having an unusually large number of expert teachers, teachers with other capacities or experience, or else enjoyed collective engagement in teaching and learning/leadership that might have made it more logical for them to become leaders in those activities in their school. A series of Bonferroni corrected independent samples t-tests comparing teacher years of experience, full-time status, perceptions of preparation, collaboration intensity, shared responsibility, satisfaction, learning climate, and shared decision making in teacher-only schools as compared to the others. Table 5 presents the results of this analysis. Overall, we found that teachers in teacher-only evaluation schools were, on the whole, not significantly different or were significantly lower in all of these characteristics than in principal-only evaluation and principal/teacher evaluation schools.

Table 5.

Independent Samples T-Tests of Differences in Characteristics of Teacher-Led Evaluation Schools Versus Other.

	Observation (n = 232)			Content knowledge (n = 323)			Test score analysis (n = 261)
	D	SE(D)	p ^a	D	SE(D)	p ^a	D	SE (D)	p ^a
Full time teacher	−0.240	0.029	.000***	−0.107	0.025	.000***	−0.113	0.027	.000***
Teacher years experience	−0.411	0.396	.287	−0.023	0.357	.949	−0.756	0.360	.094
Teacher feeling prepared to teach	−0.081	0.027	.002	−0.010	0.023	.830	−0.188	0.024	.000***
Teacher job satisfaction	−0.178	0.040	.000*	−0.099	0.040	.016	−0.104	0.038	.007
Teacher collaboration intensity	−0.160	0.030	.002	−0.072	0.043	.100	−0.194	0.046	.000***
Learning climate	−0.142	0.022	.000***	−0.060	0.018	.001*	−0.022	0.019	.336
Shared responsibility and decision making	0.022	0.027	.412	0.036	0.023	.120	0.048	0.024	.045
Principal perception of climate	−0.046	0.037	.223	0.073	0.078	.350	−0.050	0.032	.069

Actual p-values reported. Significance markers (asterisks) Bonferroni adjusted. Principal final weight applied for this analysis (SCHWGT).

***

p ≤ .001. **p ≤ .01. *p ≤ .05.

Discussion

Overall, the findings demonstrate that teacher-led evaluation is associated with more positive feelings of motivation, satisfaction, as well as change in practice. This was true for all facets of the evaluation process we examined—lesson observation, assessments of teacher content knowledge, and analysis of test score data. This evidence suggests, at the very least, we consider more carefully the role that expert peers and mentors can play in the evaluation process to maximize the effects that it can have on teachers’ classroom practice, confidence, and motivation. They also lend credence to our claim that how evaluation is done matters for how teachers ultimately feel about their feedback and how it motivates (or does not motivate) change in practice. Further, our results corroborate and expand upon those found by other researchers who have urged policymakers and leaders to consider models of peer coaching and feedback as a potentially beneficial approach to teacher evaluation (Donaldson, 2021; Galey-Horn & Woulfin, 2021; Johnson, 2019; Johnson et al., 2010; Ridge & Lavigne, 2020).

They also support the findings of other studies which have claimed that assessments of content knowledge and analysis of test scores, because they are context specific, might demand specialized knowledge that administrators might lack in order for the discussions to be beneficial for use in instructional improvement (Kraft & Gilmour, 2016; E. C. Smith et al., 2020). The nature of such discussions seem also to be more time-demanding in order for them to be done well, which, given that principals have incredibly busy schedules and struggle with the time demands of teacher supervision and evaluation (Flores & Derrington, 2017), might be best left to mentors and fellow teachers. Furthermore, the results of the use of test score data in evaluation support past findings critical of now-questionable practices like the use of VAM results which many states have discontinued because of their inherent measurement problems and the visceral reactions of educators to their use (Close et al., 2020; Ford et al., 2017).

Finally, there is the question of the effectiveness of extrinsic motivational tools in accomplishing the main purposes of evaluation. Our study found that principals’ reported use of these tools (either rewards such as increased pay or punishments such as censure, dismissal, or change of job responsibilities) to reward or punish teachers based upon their evaluation results was associated with a modest yet significantly lower motivation and reports of positive change in practice among teachers. While removing ineffective teachers can play an important role in evaluation policy (censure, demotion, reduced pay, and termination), the accumulating evidence—including that presented in this study—suggests that they might be harmful to the intrinsic motivation that a majority of teachers across the world have in abundance (Brookhart & Freeman, 1992; Chong & Low, 2009; Donaldson, 2021; Ford et al., 2017; König & Rothland, 2012; Krecic & Grmek, 2005; Kyriacou & Coulthard, 2000; Pallas, 2021; Sinclair, 2008; Watt & Richardson, 2008).

Implications for Policy and Practice

Our findings suggest, at the very least, that different teacher evaluation and supervision activities might be led by different individuals within the school; thus policymakers and practitioners may want to consider policies and practices that honor that, while administrators bring important school-wide perspectives and knowledge to understanding global aspects of instructional practice and student survey data, teachers and mentors may be able to supplement this expertise in content-based ways that support teacher growth and development, and ultimately student learning.

While not central to our guiding research questions on the relationship of evaluators to important changes in teacher motivation and practice, one notable and consistent finding from our study is the negative relationship between a principal’s use of extrinsic motivational tools after evaluation, such as rewards and consequences, and teacher outcomes (e.g., practice/motivation). Scholars have warned about the negative ramifications of high-stakes evaluation (Ford et al., 2018; Guenther, 2021; Holloway et al., 2017; Lavigne, 2014; W. C. Smith & Holloway, 2020) where teacher evaluation outcomes are tied to extrinsic rewards such as promotion, salary increase, etc., particularly on factors such as teacher affective states, motivation, retention, and school culture and climate. These findings suggest that policies and practices in favor of teacher evaluation systems that prioritize teacher growth and development may be more effective in fostering teachers’ motivation and practice than those models that prioritize holding teachers accountable using high stakes (Ford & Hewitt, 2020; Guenther, 2021; Lavigne & Good, 2020).

Limitations

While we provide theory-based explanations for the results we found, one limitation to the overall study is our inability to provide direct evidence of the mechanism—in other words, how administrator versus teacher-led evaluation results in different outcomes for teachers. For example, we do not know whether or not rich content-based discussions happen when mentors and teachers are leading content-based assessments of knowledge and analysis of test score data. Another limitation is that TALIS2013 data were collected from teachers at the lower secondary level. Perhaps content-based expertise plays a larger role in lower and upper secondary as teachers are more likely to be subject-area specialists. However, this may depend largely on the country or site. For example, as it pertains to the US, elementary teachers often teach in multiple subject areas, but in other countries, elementary-level teachers do have a content area specialty. Thus, the differential role of content may be salient in some places and insignificant in others.

Suggestions for Future Research

To better support states and districts in their use of best practices as it pertains to teacher supervision and evaluation, it would be useful for educational researchers to partner with districts willing to pilot the use of teacher-led evaluation models alongside principal-led evaluation models with the use of randomly assigned schools and teachers in a low-stakes evaluation environment. This would provide an important causal methodological extension to the analysis and research provided here which is primarily correlational.

Including direct measures of feedback from teacher-led and administrator-led components would help illuminate the how and the why. This is particularly important for understanding how teachers and mentors may or may not mitigate the challenges principals face especially as it pertains to content (Kraft & Gilmour, 2016). Outcome measures such as teachers’ perceptions of the usefulness of feedback from teacher- and administrator-led components would be pertinent (Cherasaro et al., 2016) as well as direct measures of teaching and learning especially since current teacher evaluation models have had null effects (Kim & Sun, 2021; Lavigne & Good, 2019). Finally, as supervision continues to travel under the guise of evaluation and few state models distinguish between supervision and evaluation, it would be useful to determine if the inclusion of teacher- and mentor-led evaluations help untangle these two goals and processes (Glanz & Hazi, 2019; Hazi, 1994; Mette et al., 2020).

While we recognize that teacher evaluation often serves dual roles—a summative judgment of a teacher’s effectiveness and formative feedback to improve teaching and learning (Ford & Hewitt, 2020)—the greatest value offered by teacher evaluation to teachers is often the feedback that occurs throughout the year as teachers have immediate opportunities to integrate such feedback into their daily practice. Recognizing this, we believe it is important to better understand and continue to support research that furthers our understanding of the who and how evaluation for teacher practice, motivation, growth, and development.

Footnotes

Appendix A

Table A1.

Item-Level Information on Rasch Measure Positive Change in Teacher Motivation and Practice.

Concerning the feedback you have received at this school, to what extent has it directly led to a positive change in any of the following? (TT2G30)
Item (4-point scale, no positive change to a large change)	D	Infit	Outfit
Your classroom management practices (TT2G30H)	0.50	1.03	1.04
Your use of student assessments to improve student learning (TT2G30L)	0.27	1.06	1.07
Your job satisfaction (TT2G30M)	0.02	0.92	0.90
Your teaching practices (TT2G30J)	−.01	0.90	0.90
Your motivation (TT2G30N)	−.10	0.96	0.93
Your confidence as a teacher (TT2G30F)	−.68	1.12	1.11

Note. D = item difficulty. Person separation reliability = 0.87; item reliability = 1.00. TALIS teacher questionnaire item numbers in parentheses.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

ORCID iD

Timothy G. Ford

Author Biographies

Timothy G. Ford is an associate professor of educational leadership and policy studies at the University of Oklahoma. He examines the role of school leadership in facilitating more collaborative work environments among teachers and in supporting teacher and leaders’ psychological needs as learners.

Alyson L. Lavigne is an associate professor of instructional leadership at Utah State University. Using her training as an educational psychologist and classroom researcher, Lavigne has conducted research on teacher retention, teachers’ beliefs, teacher supervision and evaluation, and culturally and linguistically diverse minoritized students’ experiences.

References

Bobbitt

(1913). Some general principles of management applied to the problems of city-school systems. Teachers College Record, 14(6), 7–96.

Brookhart

S. M.

Freeman

D. J.

(1992). Characteristics of entering teacher candidates. Review of Educational Research, 62, 37–60.

Cherasaro

T. L.

Brodersen

R. M.

Reale

M. L.

Yanoski

D. C.

(2016). Teachers’ responses to feedback from evaluators: What feedback characteristics matter? (REL 2017-190). Regional Educational Laboratory Central.

Chong

Low

E. L.

(2009). Why I want to teach and how I feel about teaching—Formation of teacher identity from pre-service to the beginning teacher phase. Educational Research for Policy and Practice, 8, 59–72.

Amrein-Beardsley

Collins

(2020). Putting teacher evaluation systems on the map: An overview of state’s teacher evaluation systems post–Every Student Succeeds Act. Education Policy Analysis Archives, 28, 58. https://doi.org/10.14507/epaa.28.5252

Cohen

D. K.

(2011). Teaching and its predicaments. Harvard University Press.

Cohen

Loeb

Miller

L. C.

Wyckoff

J. H.

(2020). Policy implementation, principal agency, and strategic action: Improving teaching effectiveness in New York City middle schools. Educational Evaluation and Policy Analysis, 42(1), 134–160. https://doi.org/10.3102/0162373719893338

Danielson

(2010). Evaluations that help teachers learn. The Effective Educator, 68(4), 35–39. http://www.ascd.org/publications/educational-leadership/dec10/vol68/num04/Evaluations-That-Help-Teachers-Learn.aspx

De Leeuw

E. D.

Hox

J. J.

Huisman

. (2003). Prevention and treatment of item nonresponse. Journal of Official Statistics, 19, 153–176.

10.

Dewey

(1929). The sources of a science of education. Liveright.

11.

Donaldson

M. L.

(2021). Multidisciplinary perspectives on teacher evaluation: Understanding the research and theory. Routledge.

12.

Donaldson

M. L.

Firestone

(2021). Rethinking teacher evaluation using human, social, and material capital. Journal of Educational Change, 22, 501–534.

13.

Donaldson

M. L.

Woulfin

(2018). From tinkering to going “rogue”: How principals use agency when enacting new teacher evaluation systems. Educational Evaluation and Policy Analysis, 40(4), 531–556.

14.

Dynarski

(2016, December 8). Teacher observations have been a waste of time and money. The Brookings Institution. https://www.brookings.edu/research/teacher-observations-have-been-a-waste-of-time-and-money/

15.

Enders

(2010). Applied missing data analysis. Guilford Press.

16.

Falabella

De La Vega

L. F.

(2016). Políticas de responsabilización por desempeño escolar: Un debate a partir de la literatura internacional y el caso Chileno [Accountability policies for school performance: A debate from international literature and the Chilean case]. Estudios Pedagógicos, 42(2), 395–413. https://doi.org/10.4067/S0718-07052016000200023

17.

Firestone

W. A.

Donaldson

M. L.

(2019). Teacher evaluation as data use: What recent research suggests. Educational Assessment Evaluation and Accountability, 31, 289–314.

18.

Flores

M. A.

Derrington

M. L.

(2017). School principals’ views of teacher evaluation policy: Lessons learned from two empirical studies. International Journal of Leadership in Education, 20(4), 416–431.

19.

Ford

T. G.

(2018). Pointing teachers in the wrong direction: Understanding Louisiana elementary teachers’ use of Compass high-stakes teacher evaluation data. Educational Assessment Evaluation and Accountability, 30(3), 251–283. https://doi.org/10.1007/s11092-018-9280-x

20.

Ford

T. G.

Hewitt

(2020). Better integrating summative and formative goals in the design of next generation teacher evaluation systems. Education Policy Analysis Archives, 28, 63. https://doi.org/10.14507/epaa.28.5024

21.

Ford

T. G.

Urick

Wilson

A. S. P.

(2018). Exploring the effect of supportive teacher evaluation experiences on U.S. teachers’ job satisfaction. Education Policy Analysis Archives, 26, 59. https://doi.org/10.14507/epaa.26.3559

22.

Ford

T. G.

Van Sickle

M. E.

Clark

L. V.

Fazio-Brunson

Schween

D. C.

(2017). Teacher self-efficacy, professional commitment, and high-stakes teacher evaluation policy in Louisiana. Educational Policy, 31(2), 202–248. https://doi.org/10.1177/0895904815586855

23.

Gabriel

Woulfin

(2017). Making teacher evaluation work: A guide for literacy teachers and leaders. Heinemann.

24.

Gagnon

D. J.

Hall

E. L.

Marion

(2017). Teacher evaluation and local control in the US: An investigation into the degree of local control afforded to districts in defining evaluation procedures for teachers in non-tested subjects and grades. Assessment in Education: Principles, Policy and Practice, 24(4), 489–505. https://doi.org/10.1080/0969594x.2016.1167669

25.

Galey-Horn

Woulfin

S. L.

(2021). Muddy waters: The micropolitics of instructional coaches’ work in evaluation. American Journal of Education, 127(3), 441–470. https://doi.org/10.1086/713827

26.

Glanz

Hazi

(2019). Shedding light on the phenomenon of supervision traveling incognito: A field’s struggles for visibility. Journal of Educational Supervision, 2(1), 1–21.

27.

Goldring

Grissom

J. A.

Rubin

Neumerski

C. M.

Cannata

Drake

Schuermann

(2015). Make room value added: Principals’ human capital decisions and the emergence of teacher observation data. Educational Researcher, 44(2), 96–104.

28.

Goldstein

(2007). Easy to dance to: Solving the problems of teacher evaluation with peer assistance and review. American Journal of Education, 113(3), 479–508.

29.

Guenther

A. R.

(2021). “It should be helping me improve, not telling me I’m a bad teacher”: The influence of accountability-focused evaluations on teachers’ professional identities. Teaching and Teacher Education, 108, 1–10.

30.

Harvey

M. W.

Boyland

L. G.

Quick

M. M.

(2019). An investigation of teacher evaluation practice in Indiana: PL 90 implementation and issues for administrators. International Journal of Educational Reform, 28(1), 24–47.

31.

Hazi

H. M.

(1994). The teacher evaluation-supervision dilemma: A case of entaglements and irreconcilable differences. Journal of Curriculum and Supervision, 9(2), 195–216.

32.

Hewitt

K. K.

(2015). Educator evaluation policy that incorporates EVAAS value-added measures: Undermined intentions and exacerbated inequities. Education Policy Analysis Archives, 23, 76. https://doi.org/10.14507/epaa.v23.1968

33.

Hill

H. C.

Blunk

M. L.

Charalambous

C. Y.

Lewis

J. M.

Phelps

G. C.

Sleep

Ball

D. L.

(2008). Mathematical knowledge for teaching and the mathematical quality of instruction: An exploratory study. Cognition and Instruction, 26, 430–511.

34.

Holloway

Sørensen

T. B.

Verger

(2017). Global perspectives on high-stakes teacher accountability policies: An introduction. Education Policy Analysis Archives, 25, 85. https://doi.org/10.14507/epaa.25.3325

35.

Hosic

J. F.

(1920). The democratization of supervision. School and Society, 45, 331–336.

36.

Howell

W. G.

(2015). Results of President Obama’s race to the top. Education Next, 15(4). http://educationnext.org/results-president-obama-race-to-the-top-reform/

37.

Hox

J. J.

(2010). Multilevel analysis: Techniques and applications (2nd ed.). Routledge.

38.

Jiang

J. Y.

Sporte

S. E.

Luppescu

(2015). Teacher perspectives on evaluation reform: Chicago’s REACH students. Educational Researcher, 44, 105–116.

39.

Johnson

S. M.

(2019). Where teachers thrive: Organizing schools for success. Harvard Education Press.

40.

Johnson

S. M.

Papay

J. P.

Fiarman

S. E.

Munger

M. S.

Qazilbash

E. K.

(2010). Teacher to teacher: Realizing the potential of peer assistance and review. Center for American Progress.

41.

Keith

T. Z.

(2015). Multiple regression and beyond: An introduction to multiple regression and structural equation modeling (2nd ed.). Routledge.

42.

Kennedy

(2005). Inside teaching: How classroom life undermines reform. Hersarvard Univity Press.

43.

Kim

Sun

(2021). The implementation and potential effects of teacher evaluation under local control. School Effectiveness and School Improvement, 32, 279–305. https://doi.org/10.1080/09243453.2020.1841657

44.

König

Rothland

(2012). Motivations for choosing teaching as a career: Effects on general pedagogical knowledge during initial teacher education. Asia-Pacific Journal of Teacher Education, 40(3), 289–315.

45.

Kraft

M. A.

Christian

(2022). Can teacher evaluation systems produce high-quality feedback? An administrator training field experiment. American Educational Research Journal, 59(3), 500–537.

46.

Kraft

M. A.

Gilmour

(2016). Can principals promote teacher development as evaluators? A case study of principals’ views and experiences. Educational Administration Quarterly, 52(5), 711–753.

47.

Kraft

M. A.

Gilmour

A. F.

(2017). Revisiting The Widget Effect: Teacher evaluation reforms and the distribution of teacher effectiveness. Educational Researcher, 46(5), 234–249.

48.

Krecic

M. J.

Grmek

M. I.

(2005). The reasons students choose teaching professions. Educational Studies, 31, 265–274.

49.

Kunter

Holzburger

(2014). Loving teaching: Research on teachers’ intrinsic orientations. In Richardson

P. W.

Karabenick

S. A.

Watt

H. M. G.

(Eds.), Teacher motivation: Theory and practice (pp. 83–99). Routledge.

50.

Kyriacou

Coulthard

(2000). Undergraduates’ views of teaching as a career choice. Journal of Education for Teaching, 26, 117–126.

51.

Lavigne

A. L.

(2014). Exploring the intended and unintended consequences of high-stakes teacher evaluation on schools, teachers, and students. Teachers College Record, 116(1), 1–29.

52.

Lavigne

A. L.

Chamberlain

R. W.

(2017). Teacher evaluation in Illinois: School leaders’ perceptions and practices. Educational Assessment, Evaluation and Accountability, 29(2), 179–209.

53.

Lavigne

A. L.

Good

T. L.

(2015). Improving teaching through observation and feedback: Going beyond state and federal mandates. Routledge.

54.

Lavigne

A. L.

Good

T. L.

(2019). Enhancing teacher education, development, and evaluation: Lessons learned from educational reform. Routledge.

55.

Lavigne

A. L.

Good

T. L.

(2020, April 29). Addressing teacher evaluation appropriately: A practice brief for school board members, superintendents, and principals. Division 15: American Psychological Association. https://apadiv15.org/wp-content/uploads/2020/04/D15-Practice-Brief-Teacher-Evaluation.pdf

56.

Linacre

J. M.

(2014). WINSTEPS rasch measurement (Version 3.81.0). Winsteps.com.

57.

Longchamp

J. C.

(2017). The effect of student learning objectives on teachers and teaching as part of the teacher evaluation process: A grounded theory study [Unpublished doctoral dissertation]. University of Vermont.

58.

Lortie

(1975). Schoolteacher: A sociological study. University of Chicago Press.

59.

Marsh

J. A.

Bush-Mecenas

Strunk

K. O.

Lincove

J. A.

Huguet

(2017). Evaluating teachers in the Big Easy: How organizational context shapes policy responses in New Orleans. Educational Evaluation and Policy Analysis, 39(4), 539–570. https://doi.org/10.3102/0162373717698221

60.

Mette

I. M.

Aguilar

Wieczorek

(2020). A thirty state analysis of teacher supervision and evaluation systems in the ESSA era. Journal of Educational Supervision, 3(2), 105–135. https://doi.org/10.31045/jes.3.2.7

61.

Murphy

Hallinger

Heck

R. H.

(2013). Leading teacher evaluation: The case of missing clothes. Educational Researcher, 42(6), 349–354.

62.

National Council on Teacher Quality. (2018). Making a difference: Six places where teacher evaluation systems are getting results. https://files.eric.ed.gov/fulltext/ED590763.pdf

63.

Newlon

J. H.

(1923). Attitude of the teacher toward supervision. National Educational Association (NEA) Proceedings, 61, 546–549.

64.

Organization of Economic Co-Operation and Development [OECD]. (2013). Teaching and learning survey 2013: Conceptual framework. https://www.oecd.org/education/school/TALIS%20Conceptual%20Framework_FINAL.pdf

65.

Organization of Economic Co-Operation and Development [OECD]. (2014a). TALIS 2013 technical report. http://www.oecd.org/edu/school/TALIS-technical-report-2013.pdf

66.

Organization of Economic Co-Operation and Development [OECD]. (2014b). TALIS 2013 results: An international perspective on teaching and learning. https://doi.org/10.1787/9789264196261-en

67.

Pallas

A. M.

(2021). The rhetoric of teacher evaluation: New York City teachers’ responses to performance labels. Educational Policy. Advance online publication. https://doi.org/10.1177/08959048211049432

68.

Papay

J. P.

Johnson

S. M.

(2012). Is PAR a good investment? Understanding the costs and benefits of teacher peer assistance and review programs. Educational Policy, 26(5), 696–729.

69.

Payne

W. H.

(1875). Chapters on school supervision: A practical treatise on superintendency: Grading: Arranging courses of study: The preparation and use of blanks, records, and reports: Examination for promotion, etc. Van Antwerp Bragg and Company.

70.

Range

B. G.

Scherz

Holt

C. R.

Young

(2011). Supervision and evaluation: The Wyoming perspective. Educational Assessment Evaluation and Accountability, 23, 243–265.

71.

Reddy

L. A.

Dudek

C. M.

Peters

Alperin

Kettler

R. J.

Kurz

(2018). Teachers’ and school administrators’ attitudes and beliefs of teacher evaluation: A preliminary investigation of high poverty school districts. Educational Assessment Evaluation and Accountability, 30, 47–70.

72.

Ridge

B. L.

Lavigne

A. L.

(2020). Improving instructional practice through peer observation and feedback: A review of the literature. Education Policy Analysis Archives, 28, 61.

73.

Roth

(2014). Antecedents and outcomes of teachers’ autonomous motivation: A selfdetermination theory analysis. In Richardson

P. W.

Karabenick

S. A.

Watt

H. M. G.

(Eds.), Teacher motivation: Theory and practice (pp. 36–51). Routledge.

74.

Ryan

R. M.

Brown

K. W.

(2005). Legislating competence: The motivational impact of high-stakes testing as an educational reform. In Dweck

Elliot

(Eds.), Handbook of competence and motivation (pp. 354–372). Guilford Press.

75.

Ryan

R. M.

Deci

E. L.

(2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Press.

76.

Ryan

R. M.

Weinstein

(2009). Undermining quality teaching and learning: A self-determination theory perspective on high-stakes testing. Theory and Research in Education, 7, 224–233.

77.

Sinclair

(2008). Initial and changing student teacher motivation and commitment to teaching. Asia-Paciﬁc Journal of Teacher Education, 36, 79–104.

78.

Singh

(2004). Tackling measurement problems with Item Response Theory: Principles, characteristics, and assessment, with an illustrative example. Journal of Business Research, 57, 184–208.

79.

Smith

E. C.

Starratt

G. K.

McCrink

C. L.

Whitford

(2020). Teacher evaluation feedback and instructional practice self-efficacy in secondary school teachers. Educational Administration Quarterly, 56(4), 671–701.

80.

Smith

W. C.

(2014). The global transformation toward testing for accountability. Education Policy Analysis Archives, 22, 116. https://doi.org/10.14507/epaa.v22.1571

81.

Smith

W. C.

Holloway

(2020). School testing culture and teacher satisfaction. Educational Assessment Evaluation and Accountability, 32(4), 461–479.

82.

Smith

W. C.

Kubacka

(2017). The emphasis of student test scores in teacher appraisal systems. Education Policy Analysis Archives, 25, 86. https://doi.org/10.14507/epaa.25.2889

83.

Stecher

B. M.

Holtzman

D. J.

Garet

M. S.

Hamilton

L. S.

Engberg

Chambers

(2018). Improving teaching effectiveness. Final report. The intensive partnerships for effective teaching through 2015–2016. https://www.rand.org/content/dam/rand/pubs/research_reports/RR2200/RR2242/RAND_RR2242.pdf

84.

Taylor

F. W.

(1911). The principles of scientific management. Harper and Brothers.

85.

Tyack

D. B.

(1974). The one best system: A history of American education. Harvard University Press.

86.

U.S. Department of Education. (2015). Every student succeeds act. https://www.ed.gov/essa

87.

Walsh

Joseph

Lakis

Lubell

(2017). Running in place: How new teacher evaluations fail to live up to promises. National Council on Teacher Quality. http://www.nctq.org/dmsView/Final_Evaluation_Paper

88.

Watt

H. M. G.

Richardson

P. W.

(2008). Motivations, perceptions, and aspirations concerning teaching as a career for different types of beginning teachers. Learning and Instruction, 18, 408–428. https://doi.org/10.1016/j.learninstruc.2008.06.002

89.

Woulfin

S. L.

Donaldson

M. L.

Gonzales

(2016). District leaders’ framing of educator evaluation policy. Educational Administration Quarterly, 52(1), 110–143. https://doi.org/10.1177/0013161x15616661