Multisource Principal Evaluation Data

Abstract

Purpose: A relatively new approach to principal evaluation is the use of multisource feedback, which typically entails a leader’s self-evaluation as well as parallel evaluations from subordinates, peers, and/or superiors. However, there is little research on how principals interact with evaluation data from multisource feedback systems. This article explores how principals orient and react to multisource feedback on their effectiveness as instructional leaders and how they interpret gaps between their self-assessments of their leadership effectiveness and their teachers’ ratings of their leadership effectiveness. Research Methods: Using interview data collected from 14 principals in an urban school district in the southeast United States at two points in time, this study conducts a qualitative analysis to examine principals’ orientations and reactions to their feedback. Findings: Our study finds that principals often experience cognitive dissonance when feedback from different data sources (e.g., their self-ratings to those of their teachers) contrasts. This can result in a motivation to reduce dissonance either by providing explanations and excuses, or making actual changes that result in improvement. Implications for Research and Practice: As performance feedback continues to become more commonplace in school settings, it will become increasingly necessary to build capacity around the processes of giving and receiving feedback. Results from this study have implications for how principals can be supported to use their evaluation data.

Keywords

principal evaluation feedback instructional leadership development leadership effectiveness

In today’s schools, the stakes for school leaders are high to meet the diverse needs of all students in a climate of system-wide accountability (Catano & Stronge, 2006; Thomas, Holdaway, & Ward, 2000). Many accountability approaches include an emphasis on principal evaluation as a lever for developing and supporting effective leadership. Principal evaluation data are used to assess principals’ strengths and weaknesses in making decisions about contracts, merit pay, and ongoing professional development.

Much of the current debate surrounding principal evaluation pertains to the extent to which school-level value added student achievement growth should be used to evaluate principals (Grissom, Kalogrides, & Loeb, 2012). Some state legislatures, such as Florida and Tennessee, mandate that student growth must be part of a principal’s performance evaluation. However, following the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999) standards, more and more principal evaluation systems require multiple measures; value added to student achievement is one measure, rating scales and leadership rubrics, for example, are others.

A relatively new approach to principal evaluation, which is well developed and implemented in other industries outside of education, is the use of multisource feedback. This approach typically entails a self-evaluation of the leader as well as parallel evaluations from subordinates, peers, and/or superiors; when all three sources are engaged this is known as 360-degree feedback.¹ The motivation behind multisource feedback is that more information regarding leadership efficacy resides within the shared experiences of these individuals than from any one source alone (Atwater, Ostroff, Yammarino, & Fleenor, 1998). Research in the private sector supports the use of subordinate feedback to facilitate communication (Boyd & Jensen, 1972), provide unique perspectives (Church, 1997), and serve as a reliable source of useful information (Smither, London, & Reilly, 2005).

There is little research on multisource feedback for principals and principals’ reactions to evaluation data from multisource feedback systems (Vohra & Singh, 2005). In an experiment from the 1960s, Daw and Gage (1967) found that feedback from teachers improves the behavior of principals. The purpose of this article is to explore how principals understand and respond to multisource feedback on their effectiveness as instructional leaders and how they interpret and react to gaps between their self-assessments of their leadership effectiveness and their teachers’ ratings of their leadership effectiveness, using multiple interviews across cases of school principals (Yin, 2009).

This topic is especially timely given the new evaluation context in schools for educators across the country. Many teachers are now evaluated using multisource feedback as their evaluations include student perceptions and ratings of their teaching. Similarly, principal evaluation is undergoing sweeping changes. Almost all states have redesigned their principal evaluation systems to include multiple measures and indicators, often in a highly politicized environment. Many of these systems rely on student achievement and growth, and additional information about principals’ instructional leadership. Some systems require surveys and ratings of principals (Goldring & Jones, 2013). There is little research about the usefulness of these systems for educator development and support, and less information about how principals react to and process performance-based feedback, even though these systems are widely assumed useful (see Murphy, Hallinger, & Heck, 2013). As multisource evaluation becomes more widespread, and principal evaluation becomes more high-stakes, research is needed to understand how principals make sense of and react to feedback.

Theoretical Framework of Multisource Feedback

Feedback intervention theory (Kluger & DeNisi, 1996) and control theory (Carver & Scheier, 1982) suggest that behavior is altered or regulated when feedback is received and compared with goals or standards. The fundamental premise underlying these theories is that dissonant cognitions induce a psychologically uncomfortable state of arousal that provides the motivation to reduce dissonance (Festinger, 1957). A discrepancy between behavior and a standard of expected performance, such as an internally defined expectation of oneself around leadership effectiveness, can increase motivation to reduce the dissonance by bringing evaluations from others in line with self-view. Research on multisource feedback suggests that multiple reports of the effectiveness of instructional leadership can expose the domains of leadership behavior most in need of improvement (Church, 1997). While self-evaluations are inextricably tied to the leader’s own experiences and biases, multiple perspectives can create contrasts to motivate behavioral change (Bickman, 2008; Riemer & Bickman, 2011) because of the importance of self-awareness and cognitive dissonance.

The comparison between self-ratings and feedback from others can challenge behavioral patterns and provide motivation to rethink behavior and its impact on others (McCauley & Moxley, 1996). Previous research shows that the reactions to multisource feedback are highly variable, with the outcome conditioned by the particular individual and situation (Walker et al., 2010). The way that persons spontaneously engage in sense-making and decision-making may lead them toward successful behavioral change or away from it.

Furthermore, the management literature on multisource ratings emphasizes the importance of understanding self-other ratings in terms of whether there is agreement, or whether leaders tend to be overestimators or underestimators because of the importance of self-awareness in understanding leadership performance (Sosik & Megerian, 1999). Other research similarly documents that self-ratings tend to be high and may be the result either of egocentric biases or because of “something unique and valid” (Conway & Huffcutt, 1997, p. 349), such as self-knowledge about behaviors that only the respondent (self) has. Research has explored the implications of over-and underestimation on performance and leadership. Findings, for example, suggest that underestimators are associated with mixed reviews by others (Sosik & Megerian, 1999); effective leaders who are self-aware are better able to incorporate observations to generate behavioral change: “Consequently, the self-aware individual is more cognizant of how he or she is perceived by others, which results in more accurate self-assessment” (Atwater & Yammarino, 1992, p. 143). From an evaluative standpoint self-awareness may be regarded as a desirable, even essential, component of a capable leader.

By and large, in the current context, principals’ source of formal evaluation feedback remains almost exclusively from the superintendent (see Goldring, Cravens, et al., 2009). This is paradoxical given that much of the research indicates it is the teachers’ perceptions of their principal’s leadership that relate to effective teaching practices and student achievement (Anderson, 1991; Goddard, Neumerski, Goddard, Salloum, & Berebitsky, 2010; Hallinger & Heck, 1996; Thomas et al., 2000). Moreover, until recently, evaluation and feedback for principals has rarely focused on matters of teaching and learning. In a recent review of evaluation practices of principals in large cities across the United States, Goldring, Cravens, et al. (2009) found very limited coverage on leadership behaviors related to curriculum or instruction and teaching, factors linked to school-wide improvement.

Feedback from teachers, in theory then, can play three key roles in the evaluation process. First, teacher feedback may provide an alternative perspective on principal instructional leadership. This “bottom-up” view of leadership may be a more valid evaluative source than the principal’s self-evaluation (Church, 1997, p. 986). The larger number of respondents also lends credence to teacher feedback representing a more reliable measure than a principal’s self-evaluation (Smither et al., 2005). Second, teacher evaluations of principals become a metric against which principals and superiors can compare and contrast the principal’s self-evaluation. Principals who consistently express views of leadership that are not supported by their faculty may reevaluate their leadership practices. Superintendents and district officials may also want to incorporate a comparison of principal and teacher ratings when considering performance reviews, or when they are recommending coaching or professional development opportunities. Furthermore, research suggests the importance of trust in developing effective leadership (Louis, Leithwood, Wahlstrom, & Anderson, 2010; Supovitz, Sirinides, & May, 2010). As noted by Price (2012), trusting, cooperative, and open characteristics in schools generate higher levels of satisfaction, cohesion around school goals, and commitment among faculty. One avenue for developing trust and openness is for leaders to know their teachers’ views or perceptions of their leadership effectiveness. As Price (2012) explains, “Sharing a definition of expectations is especially important for successful relationship outcomes” (p. 65). Collecting feedback from teachers is one possible avenue for leaders to begin to engage in open communication with teachers about their leadership practices. Third, teachers can provide principals with crucial insights into their roles as instructional leaders, specifically around the domains of teaching and learning.

Kimball (2002) developed a conceptual model to analyze factors that may influence the extent to which teachers will use feedback that can also inform the discussion on how principals might respond to feedback. He suggests that the timing and frequency of feedback, the credibility of those giving feedback, and the extent to which the feedback is aligned with the system so that feedback can be integrated and acted on are important factors in understanding reactions to feedback. Furthermore, Kimball and others refer to the importance of perceived fairness or perceived validity as important factors in reactions to feedback. Moreover, feedback, the source of dissonance, must be accepted as valid to thwart natural tendencies to self-enhance or self-protect (Alicke & Sedikides, 2009).

However, these factors that may affect the extent to which one will use feedback may be strongly influenced by principals’ “self-protective” defense reactions and rationalization that are typical psychological responses to anxiety (Vohra & Singh, 2005). Vohra and Singh (2005) found classic forms of resistance to negative feedback (ignoring and denial, repression and rationalization) when administering feedback to principals. This raises important questions regarding how principals understand feedback and how they react to gaps between their self-assessments and their teachers’ ratings of their leadership effectiveness.

Employees may fail to benefit from feedback for several reasons. First, employees are often self-enhancing and self-protective (Alicke & Sedikides, 2009) and tend to “accept feedback from others that is consistent with the way we see ourselves and to reject feedback that is inconsistent with the way we see ourselves” (Goldsmith, 2004). Mone and London (2010) observed that even prior to implementing 360-degree feedback, managers “invariably” provided rationalizations for why it would be unable to produce valid feedback given their particular circumstances. By denying the validity of the feedback, employees relieve themselves of the responsibility for responding to the feedback. When asked to reflect back on multisource feedback most remember the positive parts rather than the negative (Smither et al., 2005).

Second, employees often struggle with interpreting feedback and drawing valid conclusions (Cannon & Witherspoon, 2005). While multisource feedback reports may indicate to feedback recipients that they are not evaluated positively, the reports are often sufficiently abstract that recipients are left with little or no clarity about what they would need to do differently (Kaiser & Craig, 2005).

Finally, even employees who figure out what raters want them to do differently may not know how to design effective goals, action plans, or a process that would enable them to succeed in changing their behavior. Thus, business organizations commonly hire coaches to help managers work through the natural barriers to using feedback productively (Jarvis, Lane, & Fillery-Travis, 2006; McDowall, 2008).

The Context of the Feedback Study

As part of an ongoing research project to explore principals’ responses to feedback regarding their learning-centered leadership, principals received a feedback report three times a year. The feedback reports included responses from the principal and his/her teachers, all of whom responded to a modified version of the Vanderbilt Assessment for Leadership in Education (VAL-ED) and other measures, including principals’ and teachers’ levels of trust, and school academic focus. Bryk and Schneider (2002) explain that trust is important because of the mutual dependence that exists between school members: “All participants remain dependent on others to achieve desired outcomes and feel empowered by their efforts” (p. 41). Hoy, Hannum, and Tschannen-Moran (1998) suggest that academic press is an aspect of leadership that shows sustained influence on student outcomes over time (see Goddard, Hoy, & Hoy, 2000). The principals and their entire instructional faculty completed surveys three times over the course of the school year (fall, winter, and spring). All teacher responses were anonymous. Teacher response rates averaged 85% in this study. Research staff administered pencil-and-paper surveys to schools on professional development days or during faculty meetings to collect the feedback surveys. Data were summarized and synthesized into 11-page feedback reports, which were distributed to school principals 2 to 3 weeks after survey completion.

Feedback Reports

Feedback reports presented survey results to principals as text, graphics, and in tabular form. The report provided principals with an overview of the key measures, namely, Learning-Centered Leadership, a Culture of Trust, and Academic Focus. The feedback was designed to induce a sense of cognitive dissonance in principals by contrasting their perceptions with those of their teachers; this has been referred to in the evaluation literature on multisource feedback as the perceptional gap or levels of perceptual congruence (Atwater & Yammarino, 1992; Boyd & Jensen, 1972; Smither et al., 2005). Feedback reports were also designed to motivate principals by presenting comparisons between their teachers’ responses and responses from other teachers and principals in the district participating in the study. The reports also provided data over time, for each administration of the surveys. An example of such feedback is shown in Figure 1.² The graph shown in Figure 1 was accompanied by brief explanatory text, such as, “Your teachers’ rating of your leadership effectiveness varied: 13% of your teachers rate you as outstandingly effective and 40% rate you as minimally effective.” Parallel presentations were provided for teacher trust of other teachers, teachers’ trust of principal, principals’ trust of teachers, and academic focus (academic press). Feedback reports included longitudinal as well as distributional comparisons.

Figure 1.

Sample feedback to principal.

To mitigate the possibility that principals would be intimidated by or misunderstand the data in the feedback reports, the research staff held two professional workshops to familiarize principals with the foundation of the measures, the organization of the feedback report, and provide them with the opportunity to ask questions about the interpretation of the data. Although the feedback reports contained suggestions for how to improve in each of the featured domains, there was no input from the research staff or the district to help principals translate the feedback data into clear goals and actionable behaviors during the first three waves of the project.³

Instrumentation

The key areas, or measures, where principals received feedback were a measure of learning-centered leadership effectiveness, based on the VAL-ED survey, measures of trust, and academic press. As noted above, the principal and all the teachers in the school completed a modified version of the VAL-ED, a 72-item assessment to evaluate the principal’s effectiveness as an instructional leader three times during the academic year. The behaviors that are measured are based on the instructional leadership literature and are aligned with the Interstate School Leaders Licensure Consortium (Goldring, Porter, Murphy, Elliot, & Cravens, 2009; Porter et al., 2010a, 2010b). The VAL-ED assessment measures six core components and six key processes that define learning-centered leadership (Goldring, Cravens, et al., 2009). The core components are high standards for student learning, rigorous curriculum, quality instruction, culture of learning and professional behavior, connections to external communities, and performance accountability. The key processes are planning, implementing, supporting, communicating, monitoring, and advocating (see Goldring, Porter, et al., 2009; Murphy, Elliott, Goldring, & Porter, 2007). The respondents were asked to rate the principal’s effectiveness for each behavior. The effectiveness rating is a scale from 1 to 5: 1 = ineffective, 2 = minimally effective, 3 = satisfactorily effective, 4 = highly effective, 5 = outstandingly effective. The teacher version includes a sixth response option, “don’t know,” for each item.

The VAL-ED has undergone extensive psychometric development and testing (see Porter et al., 2010a, 2010b). The VAL-ED was developed and psychometrically tested in a field test after an extensive item writing and instrument development phase, including a sorting study, two rounds of cognitive interviews, a bias review, and two rounds of small-scale pilots. The field trial sample consisted of more than 270 schools and more than 8,000 individual surveys, with 218 complete sets of responses. The sample included urban, suburban, and rural schools; elementary, middle, and high schools; and schools from all regions of the country (Porter et al., 2009). A recent differential item functioning by Polikoff et al. (2009) shows that item functionality is robust across a wide spectrum of contextual variation including urbanicity, geographic region, and school-grade level (elementary, middle, and high schools). The Cronbach alpha reliability measure is more than .95 for both principals and teachers.

The feedback measure of teachers’ trust of their principal, adapted from extant measures (e.g., Hoy & Tschannen-Moran, 1999, 2003), includes five items (Cronbach α reliability of .89). Teachers rate the items on a 4-point Likert-type scale, ranging from “Strongly disagree” to “Strongly agree.” The measure is part of the feedback. Sense of trust has criterion validity with student achievement (Louis et al., 2010). Teacher trust items include the following: During the past few months . . . teachers in this school can rely on the principal; teachers in this school trust the principal; this principal doesn’t really tell teachers what is going on; the teachers in this school have faith in the integrity of the principal; the teachers in this school are suspicious of most of the principal’s actions.

Academic press as perceived by principals and teachers is a four-item scale (reliability of .82 for teachers and .80 for principals). Respondents rate on a 4-point scale how much they agree/disagree with such statements as the following: Teachers in this school expect students to complete every assignment, and Teachers in this school set high expectations for academic work. This construct is consistent with other work such as Lee and Smith (1999) and has predictive validity with student achievement. The data for these measures were presented to the principals in the feedback reports in a similar graphic as the VAL-ED measures.

Methodology

This article is part of a larger research project that was conducted in an urban school district in the southeastern United States that serves about 75,000 students. Sixty-five percent of students in the district are eligible for free or reduced price lunch; 34% of the students are White, 48% are Black, and 14% are Latino. English language learners account for about 12% of the students.

This article is based on a qualitative methodology that relied on a data collection strategy of one-on-one interviews during two points of time. Schools where principals were interviewed were drawn from 36 schools that were randomly assigned to a treatment group to receive a feedback report three times during the school year. Qualitative data was collected from a total of 23 schools in the broader research project. These schools were purposefully selected based on school free and reduced lunch rates, racial composition, and initial baseline teacher and principal feedback ratings. This article focuses on a narrower sample of these schools (n = 14), which had principals who both received feedback during the 2008-2009 academic year and were not new to the principalship.⁴ We decided against including brand new principals in our analysis for logistical reasons; many of the new principals who participated in our study were recruited after the school year had already begun, so it was not possible to interview them during the fall.

The interviews proceeded during two time periods. In the first stage of data collection during fall of 2008, a team of interviewers conducted brief telephone interviews with principals the week after they received their first feedback reports. These 20-minute interviews served to canvass principals and capture their initial reactions to receiving feedback. Interview questions included, “When you first read your feedback report, what was the first thing that ran through your mind?”

During the second stage of data collection during the winter of 2009, interviewers conducted in-depth in-person interviews with principals lasting up to an hour to follow-up on principals’ initial reactions to their feedback, asking questions about whether their initial reactions had changed, if they had processed their feedback further or shared it with anyone, if they had gained insights from the feedback, whether they planned to respond to the feedback (e.g., share with assistant principal, leadership team, whole faculty) or change leadership behaviors, and whether they had plans to pursue professional development activities related to their feedback. Relevant excerpts of both interview protocols are included in Appendix A. A statement of researchers’ positionality is included in Appendix B.

Analysis

The analysis focused on exploring principals’ orientations and reactions to multisource feedback on their instructional leadership effectiveness, trust and academic press, and how they interpret and react to gaps between their self-assessments and their teachers’ ratings of their effectiveness in these areas. We define principals’ orientation to feedback as the way principals viewed and felt about their feedback while reaction to feedback captures how principals respond to and engage with the information included in their feedback reports. Analyses sought to fully understanding each principal’s experience with receiving feedback, and also compare all cases to identify the range and differences among principals’ orientations and reactions to their feedback.

This analysis utilized pattern coding to discern patterns of thought, action, and behavior among principals (Fetterman, 1989; Yin, 2009). To explore the research questions, we established a baseline a priori frame from the research questions and applied it during the first round of coding the data. Interview transcripts were coded and summarized according to general descriptive categories derived from the theoretical framework and the literature review using the constant comparative method (Patton, 2001). Specifically, our coding process was guided by the key themes in the data and the concepts unpacked from the theoretical framework linked to our project questions and interview protocols (e.g., how principals orient and react to feedback on their instructional leadership, how principals feel about their feedback, principals’ perceptions of the accuracy of their feedback, how principals interpret gaps in their self-ratings and those of their teachers, etc.). As an example of our coding scheme, when examining principals’ orientation to feedback, we utilized the following codes initially: positive, neutral, and negative. We subsequently added the following codes as they emerged from the data: receptive, curious, and defensive. In addition, the extant research literature guided our specific coding scheme. For instance, when examining how principals react to feedback, we utilized Vohra and Singh’s (2005) forms of resistance to feedback (avoidance and denial, rationalization, superficial interpretation, and unnatural behavior) as specific codes.⁵ This process was both iterative and theory driven, and reflected inductive and deductive analysis (Strauss & Corbin, 1990). We also made an effort to compare interview transcripts to see if additional patterns across individuals emerged. As new categories and themes emerged during data coding, they were added to the coding framework (Miles & Huberman, 1994). Thus, the resulting theories that “bubble up” from the findings are grounded in real-world patterns (Glaser & Strauss, 1967).

The ultimate coding template included procedures that were both guided and open, with codes that were categorical and thematic. In sum, our analytic procedures were designed to produce a reliable and valid qualitative report, including an examination of countervailing evidence, constant comparative method, and iterative coding (Patton, 2001). The final analyses include thick, rich descriptions of the patterns and themes that emerged from the interview data.

Findings

The analyses of the interview data suggest that principals’ orientations and reactions to their feedback vary greatly and this variation has implications for how principals can be supported to use their evaluation data.

Orientation to Feedback

Principals’ orientation to feedback focused on how they felt they had performed in comparison to how they had expected to perform (see leftmost section of Figure 2). Interview data revealed that principals honed in on whether their data exceeded or fell below their expectations as demonstrated by three specific aspects of their feedback report: (1) a comparison between their own self-ratings of instructional leadership and academic press and the average rating of their teachers, (2) a comparison between their teachers’ average instructional leadership and academic press rating and that of other principals in the district, and (3) a comparison between their teachers’ average rating of how much they trust the principal and that of other principals in the district. These comparisons allowed principals to know how they were doing in their school building as well as nest their results in the broader district context. Of these comparisons, principals tended to focus the most on the difference between their self-rating of instructional leadership and their teachers’ average rating in this area. Specifically, three out of four principals whose instructional leadership self-ratings were noticeably less than their teachers’ ratings were positive and receptive to feedback, six out of seven principals whose self-ratings were approximately equal to their teachers’ ratings were neutral and curious or positive and receptive, and two out of three principals whose self-ratings were noticeably greater than their teachers’ ratings were negative and defensive.⁶

Figure 2.

Framework for orientation and reaction to feedback.

Principals whose teacher ratings met or exceeded their own self-ratings and/or the ratings of other principals in the district generally had a neutral or even positive orientation to feedback. For example, principals whose ratings exceeded their expectations were receptive to their feedback:

I thought it reflected, probably, a good reflection of where I felt we had been this past year, and a pretty honest reflection of where the teaching staff was, and so there were no big surprises. I thought it was a pretty accurate response, on both of our parts. (Principal 1079a)

Another principal whose ratings met expectations described feeling “comfortable” with the contents of the feedback report:

Interviewer:

When you first read the feedback report, what was the first thing that ran through your mind?

Principal:

Just looking at the gaps between what I felt and what my staff felt. That was the first thing I looked at.

Interviewer:

Okay, and did anything stand out, particularly, to you?

Principal:

Not really, other than [the gap] wasn’t a large one. I was comfortable with, you know, that we were pretty, it seemed like to me, pretty on track with what I felt I was doing and what they thought I was doing. So it wasn’t very big, but it did give me an opportunity just to reflect on what I’m doing, what I think that I need to tweak, to address particular areas where there was a little bit of a gap.

Interviewer:

So you reviewed it, and there weren’t very many gaps, but the things that you saw, you wanted to work on?

Principal:

Right. (Principal 1125a)

Principals who received feedback that was in line with or better than what they had anticipated tended to approach their feedback with a positive orientation that in turn shaped how they subsequently engaged with feedback results.

Conversely, principals who had rated themselves higher than their teachers or whose teachers’ ratings fell below the average for the district often experienced cognitive dissonance. Interestingly, principals responded to this cognitive dissonance in two different ways. Most of these principals displayed a negative and at times defensive orientation to their feedback. For example, when asked to describe what ran through her head on receiving the feedback report, one principal whose teachers rated the principal well under the principal’s self-evaluation said “I had some real concerns about the results . . . and I have some real concerns about the perspectives of my teachers” (Principal 1059a). In essence, this principal had immediate reservations about the results that led to suspicion around the validity of the feedback.

However, a handful of principals whose self-ratings were different (lower) than their teachers or the district average developed a more neutral orientation to their feedback, displaying curiosity about the surprising results rather than adopting a negative orientation to their feedback (dotted arrow in Figure 2). For example, in examining results about academic focus, one principal started asking questions like: “Okay, how can I make this become better? How can I be the best administrator that I can possibly be? How can I create a learning environment where my students strive to be as successful as they possibly can?” (Principal 1028a) and took the time to revisit the feedback report to further engage with it: “I skimmed it the first time that I read it; then when I got back home, I read it a little closer” (Principal 1028a). In this case, despite the cognitive dissonance, this principal’s more neutral orientation to the feedback allowed the principal the space to become curious about the data and begin to ask questions that had the potential to lead to a deeper level of thought and reflection.

Principals’ orientation to feedback in many ways seemed to set the tone for how they in turn used the feedback from that point on; principals who were initially disappointed with their feedback results appeared to struggle to connect with the feedback, often distancing themselves from the issues raised in their reports, while principals who were initially eager to receive and examine feedback tended to remain that way. In essence, the orientation to feedback colored the way principals viewed and made sense of the feedback and prompted principals to adopt a particular stance toward the feedback.

Reaction to Feedback

Interview data reveal that orientation to feedback was closely tied to principals’ reaction to feedback. Principals who were either positive and receptive or neutral and curious about their feedback expressed interest and engagement when working through their feedback reports. For example, one principal whose scores had exceeded expectations and had a positive orientation toward the feedback described the first time she looked at her feedback report: “I thought it was very interesting, and I read it very quickly, because I was leaving the house, and then, when I got back, I spent a little more time on it” (Principal 1079a). Many principals with a positive orientation to their feedback chose to revisit their feedback report to further engage with the information and try to learn about what they could improve. This type of reaction to feedback often also led to principals beginning to formulate and enact plans in response to their ratings. For example, in the second wave of interviews another principal continued to come back to the feedback report for guidance:

But the areas: the two core components that were—areas as your least effective . . . the connections to external communities: that was one that I really agreed upon . . . I know that we must have both school and home working together, and I really do believe that we must—for us to do what we need to do here with our children, we must get our parents and community more involved, and so that was one of the areas that I asked our teams to work on, just with suggestions, and I did a concern sheet, as a team, and the area on the external communities. . . . So I pulled that from what you guys had given to me. (Principal 1041b)

Similarly, this same principal was able to grapple with the aspects of the report that indicate a need for improvement:

In looking at comparisons for the teachers in this building, for myself, and then district teachers and principals, that was interesting, too, to see where we’re rating with others . . . and even when I was growing up, and even with raising my children, you know, we shouldn’t compare ourselves with others, but I think that’s a natural thing to do, but so in looking at this, okay, well, we’re right in there. In some areas, we were a little above, so as a principal, that helped me to feel better. The trust factor, though, interesting enough, when I look at the district teachers and my teachers, the teachers here in this building were a little below the district, and so again, I’m thinking, “Well, what is that indicating or saying to me?” (Principal 1041b)

Principals who adopted a neutral or positive orientation toward their feedback were much more receptive to and engaged with the data contained in their report, even the data that highlighted areas that could be improved on. They were not immediately overwhelmed by cognitive dissonance as they initially worked through their feedback, which allowed them to comfortably confront the more challenging aspects of the report.

In contrast, principals who had a negative orientation often reacted adversely to their feedback. Principals whose self-ratings were either higher than those of their teachers or whose ratings were simply lower than expected in general often expressed immediate dismay with their feedback. These recipients who were displeased with their feedback results tended to display self-protecting behaviors designed to deflect responsibility for feedback results. Vohra and Singh (2005) record four typical adverse reactions to feedback: (1) avoidance and denying feedback received, (2) rationalization of the feedback, (3) superficial interpretation of the feedback, and (4) unnatural behavioral manifestations on receiving feedback. As we coded our data, we found that all of principals’ adverse reactions to their feedback fell into one of these categories.

Avoiding and denying feedback

Avoidance occurs when the feedback recipient seeks to ignore or disbelieve the information contained in the feedback (Vohra & Singh, 2005). Principals who avoided their feedback often spent little, if any, time reading and interpreting their report. For example, one principal described putting the report aside on experiencing disappointment with scores:

Principal:

Looked like, to me, when I—that my ranking was low. That was my—I don’t know if I interpreted it correctly.

Interviewer:

Um-hmm. Um-hmm . . . So how did that make you feel?

Principal:

Put it away.

Interviewer:

Put it down, you mean? Just walk away from it?

Principal:

Yeah. Yeah, exactly. (Principal 1015a)

Avoidance can also occur when feedback recipients are interpreting their feedback. For example, Vohra and Singh (2005) found that many of their feedback recipients were “cognitively lazy and wanted [the researchers] to interpret their feedback” (p. 143). The following exchange is one example of a principal who is interested in having the interviewer work through the feedback and explain it:

Interviewer:

And then the bottom graph is the district comparison.

Principal:

Yes. Overall principal, 3.6, and then what was mine? Does it—?

Interviewer:

Which overlaps; yeah, it’s overlapping.

Principal:

I see.

Interviewer:

Actually that was, this is the overall teacher average across schools, and overall principal average, and your rating was a four, so compared—

Principal:

And theirs is a 3.6.

Interviewer:

—to the district; right.

Principal:

Right. I felt like it was pretty close.

Interviewer:

Um hum . . . was 3.4. So you’re slightly ahead of the district. (Principal 1043b)

Our analysis shows that principals who avoided or denied their feedback often distanced themselves from their results by not engaging with the material contained in the report.

Rationalization of the feedback

Feedback recipients rationalize their feedback when they offer excuses as to why their feedback was below expectations (Vohra & Singh, 2005). While attempting to more deeply understand results by situating them in the school setting can be productive, there is a fine line between explanation and rationalization. Explanation seeks to shed additional light on results, whereas rationalization attempts to explain away results. One manifestation of rationalization may be to attempt to “locate the sources of the feedback” (Vohra & Singh, 2005, p. 145). For example, one principal rationalized her lower than expected scores by discussing the impact of a new teacher she hired:

Principal:

And, you know, I guess this one thing was the one that really stood out in my mind because I do have a teacher this year that I hired—4.0, great recommendations, great student teaching, great whatever, whatever, whatever. And I’m having a real difficult time seeing her fit in this environment. And it’s really making me question this and question the trust and faith that I had in her when we, she went into there. Because everything was in place on paper, everything was in place on the interview, but the problem with this particular person is she cannot get along with her peers. And it’s kind of the prima donna. And, I’ve done everything I can to work with her, encourage her, talk with her . . . kind of make her reflect at her own behaviors. And it’s just not going to happen.

Interviewer:

So that makes you question the trust, although the trust is still quite high there, but it makes you question your trust of her?

Principal:

Right. Right. Right, and it’s really sad. It’s really sad, ’cause she’s got everything going for her. But when you don’t get along with your peers, it’s like you’re bringing that attitude into your building and it’s just poison. (Principal 1043b)

Instead of talking about working to build a better relationship with this teacher, the principal talks about the negative impact of this teacher as a foregone conclusion that cannot be remedied. In addition, it was also common for principals to indicate a tendency to think about the kind of day their teachers had as they examined their feedback, suggesting that lower scores may be an indication that teachers completed the survey during a particularly hectic time of year.

Superficial interpretation of feedback

Feedback recipients who skip over subtleties of the feedback in favor of focusing only on that which is obvious are approaching their feedback in a superficial fashion (Vohra & Singh, 2005). Superficiality is often demonstrated when feedback recipients gloss over uncomfortable parts of their feedback results instead emphasizing aspects of the feedback that are positive. One such example is a principal who indicated she had started doing more goal setting and on being asked whether or not she is using any particular resources to help with these changes replied, “Just good logic” (Principal 1028a). When interpreting the lower than expected rating of teachers’ trust of other teachers in the school, one principal said:

You know, there are a whole lot of us that work—a whole lot of adults that work in this building [laughing]. It’s hard to be able to trust everyone else at all times . . . it’s never going to be a hundred percent. So, to me it’s pretty strong so I’m, you know, I’m pretty, I’m pretty good with that, again, it’s right there in the district, so . . . (Principal 1003b)

Interestingly, this principal uses the district comparison to the ratings at her own school to justify maintaining the status quo.

Unnatural manifestations

Those who exaggerate and dramatize their reactions to feedback are demonstrating unnatural manifestations (Vohra & Singh, 2005). These exaggerations can help mask true feelings about less than stellar feedback. One principal reacted dramatically on noticing that teachers in her school had indicated that the school was less academically focused as she had:

About three percent of your teachers disagree that your school has a strong academic focus. That’s about the biggest joke I’ve ever heard, you know? I mean, are you kidding? [Our School]? I mean, I have parents beating down the door to check to make sure what their children are learning, and if they’re not reading at, you know, their age level, God help them all, because there’s something seriously wrong with their child. (Principal 1053b)

Another principal expressed frustration with the learning-centered leadership score her teachers had given her as compared with her self-rating:

I think they just don’t realize how much they’re doing. And how good they are. It’s very difficult. Our reading scores are 96-97 percent. And it’s very difficult to maintain that year after year after year. And I have so many people that are such perfectionist teachers; I don’t care how much we do, they would still think there’s more, more, more, more, more out there, you know, that we’re just still not doing our best. . . . But, you know, I think it has to do with a lot too with the . . . expectations of the teachers. I have teachers that have very high expectations of themselves, and some of them, even if it was a hundred percent, their expectations, you know, would never be met. (Principal 1043b)

Responses like these can be problematic because they represent an attempt to downplay the meaning of the data. Doing so may seriously limit the potential for the feedback to promote change and improvement.

Discussion and Implications

Discussion

Against the backdrop of the accountability movement, assessment and evaluation are playing an increasingly prominent role in schools across the country. While much of the focus has been on teacher evaluation systems, policy makers have begun to shift attention to other important parties, such as school administrators. Increasingly, states are embracing evaluations systems that include multiple measures of performance. Multisource feedback is one such measure.

Unlike other measures of performance (e.g., school-level value added models that capture student achievement growth), multisource feedback is designed to develop and support feedback recipients. A primary goal of providing principals with feedback on their performance is for formative purposes; that is, on receiving timely and useful information about their performance, principals will be able to adjust and improve their practice based on what they have learned. The information included in our feedback reports was meant to help principals identify areas of strength to celebrate and areas of weakness to target for improvement. Our study reveals that while providing principals with feedback certainly has the potential to lead to improved performance, it does not guarantee it. Receiving feedback can be an emotionally taxing process because a primary purpose of multisource evaluation systems is to highlight areas of improvement that the feedback recipient may not be aware exist.

How principals understand, interpret, and process feedback is complex and multifaceted. Interviews with our principals suggest that much of how principals interpret feedback is through comparison. They compare their self-ratings to those of their teachers, their teachers’ ratings to teachers’ ratings for other principals across the district, and their ratings over time. Principals tend to focus their attention on contrasts within these comparisons, particularly instances when their teachers have rated them lower than their self-ratings, or their teachers have rated them below the district average. Such situations prompt principals to experience cognitive dissonance, which can result in a motivation to reduce dissonance either by providing explanations and excuses, or making actual changes that result in real improvement.

Results from this study show that in many ways principals’ initial interactions with their feedback guide and color the way that they process their feedback. Principals were quick to adopt an orientation toward their feedback on receiving their first report, which tended to dictate the way they reacted to their feedback. This suggests that it may be helpful to work with principals prior to receiving initial feedback to prepare them for what they may encounter once they see their results and offer some suggestions as to how to understand and process the feedback. As Kimball (2002) asserted when studying factors that influence teachers’ use of feedback, it is important that feedback recipients perceive their feedback as both fair and valid. Working with principals in advance of receiving feedback to establish the credibility of the data might help to prevent principals from adopting self-protecting behaviors and a defensive orientation toward challenging feedback, instead allowing them to take a more neutral and curious orientation toward their feedback that prompts them to wrestle with and act on the data contained in their report.

Limitations

As is the case with most studies, this research has both strengths and weaknesses. In terms of strengths, this study is able to draw on the results from principals’ own feedback reports to inform how principals understand their feedback. In essence, because we know how these principals scored in terms of self-and teacher-ratings, we are able to examine whether or not this is an important factor in understanding how principals make sense of their feedback. In addition, we interviewed multiple principal participants at two points in time providing us with divergent responses that made our data thicker than it would have been if we had interviewed a smaller group and had only done so once.

An obvious limitation of this study is the fact that the data are drawn from only one school district, and it is a subsample of principals within a broader study. That said, we believe that this district is confronting issues akin to those of other large urban school systems across the nation, both in terms of pressure regarding student performance as well as navigating newly imposed systems of accountability and evaluation. One additional limitation of this study is that while our study had district support, it was not tied to principals’ actual evaluations. The data collected in this study were for research purposes and were not provided to district personnel. Despite these limitations, we believe this work makes a valuable contribution to the research literature and has important implications for how districts approach the process of providing feedback within a system of high-stakes accountability.

In addition, research has clearly documented the importance of context in understanding school leadership. In this study, we do not unpack the ways in which context could influence both the direction of the feedback and the reactions to feedback. For example, in some cases a discrepancy might be expected if a principal was specifically hired into a turnaround situation where teachers might be disgruntled and have a low sense of efficacy. Context is an important avenue for future research in the study of feedback.

Implications

As performance feedback continues to become more commonplace in school settings, it will become increasingly important to build capacity around the processes of giving and receiving feedback. Districts that provide principals with performance feedback as part of their evaluation system expect principals to make use of that information to improve their practice. However, this study demonstrates that merely providing principals feedback, particularly feedback that conflicts with principals’ self-perceptions, and expecting them to manage their own cognitive dissonance may be unrealistic and counterproductive. Instead, it will be important for districts to consider ways to scaffold principals’ learning around how to productively manage the cognitive dissonance they experience. This will also require new training, awareness and re-visioning of the role of principal supervisors and central office leaders. In other words, supervisors will need to support and encourage more principals to understand and connect possible cognitive dissonance to orientation toward their feedback.

Districts might consider offering coaching and development workshops in which principals are provided with different mock-ups of feedback reports and asked to describe their orientations and reactions to the feedback as if it were their own. They could then provide illustrations of what it looks like when principals adopt a curious versus defensive orientation toward the feedback. Principals could also practice grappling with defensive comments by reframing them as questions that seek to glean more information about the underlying causes of the cognitive dissonance. Models such as Argyris’ advocacy and inquiry can help principals learn how to respond to feedback (Argyris, 1999). Similarly, leadership coaches can help principals develop the skills necessary to work through their feedback in a productive, forward-moving fashion (see Goff, Guthrie, Goldring, & Bickman, in press). More research is needed on how to help principals (and teachers) develop capacity to manage the psychological discomfort that often comes with challenging feedback.

Footnotes

Appendix A

Appendix B

Acknowledgements

We are grateful to our colleagues from the Improving Principal Leadership through Feedback and Coaching project, in particular, Len Bickman, Peter Goff, Jason Huff, and Mark Cannon for their collaboration and support. We also wish to recognize and thank the participants in our study for taking the time to speak with us.

Declaration of Conflicting Interests

The authors declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The Vanderbilt Assessment of Leadership in Education (VAL-ED) instrument is authored by Drs. Porter, Murphy, Goldring, and Elliott and copyrighted by Vanderbilt University, all of whom receive a royalty from its sales. The VAL-ED authors and their research partners have made every effort to be objective and data based in statements about the instrument and value the independent peer review process of their research. With any publication, readers should judge the facts and related materials for themselves.

Funding

The authors disclosed receipt of the following financial support for research, authorship, and/or publication of this article: This research was supported through a grant from the U.S. Department of Education’s Institute of Education Sciences for the Improving Principal Leadership through Feedback and Coaching project (R305A070298). Funding for Madeline Mavrogordato was provided by a grant from the U.S. Department of Education’s Institute of Education Sciences to Vanderbilt’s ExpERT program for doctoral training (R305B080025). Additional support was provided to Madeline Mavrogordato by the Education Policy Center at Michigan State University.

Notes

Author Biographies

Ellen B. Goldring is Patricia and Rodes Hart Professor of Education Policy and Leadership, and Chair, Department of Leadership, Policy and Organizations, Peabody College at Vanderbilt University. Her research interests focus on the intersection of education policy and school improvement with particular emphases on school organization, school choice, and education leadership. Her research on school leadership examines leadership practice, and the implementation and effects of professional development, coaching, and performance feedback.

Madeline Mavrogordato is an assistant professor of K-12 educational administration in the College of Education at Michigan State University. Her research utilizes both quantitative and qualitative methods to investigate how the implementation of educational policies, systems of school governance, and school leadership shape educational outcomes for underserved students. She is interested in how to prepare school leaders to meet the diverse needs of all students, particularly in a climate of system-wide accountability and evaluation.

Katherine Taylor Haynes is Assistant Director of the National Center on Scaling Up Effective Schools and a research associate in the department of Leadership, Policy, and Organizations at Peabody College, Vanderbilt University. She has extensive experience conducting qualitative research on mixed-method projects. Her research focuses on the social context of education and public policy, with specific emphasis on the characteristics of effective schools including principal leadership, the implementation of district-designed school innovations, and new teacher induction policies.

References

Alicke

M. D.

Sedikides

(2009). Self-enhancement and self-protection: What they are and what they do. European Review of Social Psychology, 20, 1-48.

American Educational Research Association, American Psychological Association, & National Council on Measurement in Education. (1999). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.

Anderson

(1991). Cognitive politics of principals and teachers: Ideological control in an elementary school. In Blase

(Ed.), The politics of life in schools: Power, conflict, and cooperation (pp. 129-138). Newbury Park, CA: Sage.

Argyris

(1999). On organizational learning (2nd ed.). Malden, MA: Blackwell Business.

Atwater

L. E.

Ostroff

Yammarino

F. J.

Fleenor

J. W.

(1998). Self-other agreement: Does it really matter? Personnel Psychology, 51, 577-598.

Atwater

Yammarino

F. J.

(1992). Does self-other agreement on leadership perceptions moderate the validity of leadership and performance predictions? Personal Psychology, 45, 141-164.

Bickman

(2008). A measurement feedback system (MFS) is necessary to improve mental health outcomes. Journal of American Academy of Child & Adolescent Psychiatry, 47, 1114-1119.

Boyd

B. B.

Jensen

J. M.

(1972). Perceptions of the first-line supervisor’s authority: A study in superior-subordinate communication. Academy of Management Journal, 15, 331-342.

Bryk

Schneider

(2002). Trust in schools: A core resource for improvement. New York, NY: Russell Sage Foundation.

10.

Cannon

M. D.

Witherspoon

(2005). Actionable feedback: Unlocking the power of learning and development. Academy of Management Executive, 19, 120-134.

11.

Carver

C. S.

Scheier

M. F.

(1982). Control theory: A useful conceptual framework for personality–social, clinical, and health psychology. Psychological Bulletin, 92, 111-135.

12.

Catano

Stronge

J. H.

(2006). What are principals expected to do? Congruence between principal evaluation and performance standards. NASSP Bulletin, 90, 221-237.

13.

Church

A. H.

(1997). Do you see what I see? An exploration of congruence in ratings from multiple perspectives. Journal of Applied Social Psychology, 27, 983-1020.

14.

Conway

J. M.

Huffcutt

A. I.

(1997). Psychometric properties of multisource performance ratings: A meta-analysis of subordinate, supervisor, peer and self-ratings. Human Performance, 10, 331-360.

15.

Daw

R. W.

Gage

(1967). Effect of feedback from teachers to principals. Journal of Educational Psychology, 58, 181-188.

16.

Festinger

(1957). A theory of cognitive dissonance. Evanston, IL: Row Peterson.

17.

Fetterman

(1989). Ethnography. Thousand Oaks, CA: Sage.

18.

Glaser

B. G.

Strauss

A. L.

(1967). The discovery of grounded theory: Strategies for qualitative research. Piscataway, NJ: Aldine Transaction.

19.

Goddard

R. D.

Hoy

W. K.

Hoy

A. W.

(2000). Collective teacher efficacy: Its meaning, measure, and impact on student achievement. American Educational Research Journal, 37(2), 479-507.

20.

Goddard

Y. L.

Neumerski

C. M.

Goddard

R. D.

Salloum

S. J.

Berebitsky

(2010). A Multilevel exploratory study of the relationship between teachers’ perceptions of principals’ instructional support and group norms for instruction in elementary schools. Elementary School Journal, 111, 336-357.

21.

Goff

Guthrie

J. E

Goldring

Bickman

(in press). Changing principals’ leadership through feedback and coaching: The results of a longitudinal intervention. Journal of Educational Administration.

22.

Goldring

Cravens

X. C.

Murphy

Porter

A. C.

Elliott

S. N.

Carson

(2009). The evaluation of principals: What and how do states and urban districts assess leadership? Elementary School Journal, 110, 19-39.

23.

Goldring

Jones

(2013). Principal evaluation systems and principals’ use of teacher effectiveness data: Polices, practices, and prospects. Paper presented at the Consortium for Research on Educational Assessment and Teaching Effectiveness, Atlanta, GA.

24.

Goldring

Porter

Murphy

Elliot

Cravens

(2009). Assessing learning centered leadership: Connections to research, standards and practice. Leadership and Policy in Schools, 8, 1-36.

25.

Goldsmith

(2004). Coaching for leadership [Online broadcast]. Retrieved from http://www.targetlearn.com/courses/course.asp?CourseID=259

26.

Grissom

J. A.

Kalogrides

Loeb

(2012, November). Using student test scores to measure principal performance (National Bureau of Economic Research Working Paper No. 18568). Retrieved from http://www.nber.org/papers/w18568

27.

Hallinger

Heck

R. H.

(1996). Reassessing the principal’s role in school effectiveness: A review of empirical research, 1980-1995. Educational Administration Quarterly, 32(1), 5-44.

28.

Hoy

W. K.

Hannum

Tschannen-Moran

(1998). Organizational climate and student achievement: A parsimonious and longitudinal view. Journal of School Leadership, 8(4), 336-359.

29.

Hoy

W. K.

Tschannen-Moran

(1999). Five faces of trust: An empirical confirmation in urban elementary schools. Journal of School Leadership, 9, 184-208.

30.

Hoy

W. K.

Tschannen-Moran

(2003). The conceptualization and measurement of faculty trust in schools: The Omnibus T-Scale. In Hoy

W. K.

Miskel

C. G.

(Eds.), Studies in leading and organizing schools (pp. 181-208). Greenwich: CT: Information Age.

31.

Jarvis

Lane

Fillery-Travis

(2006). The case for coaching: Making evidence-based decisions on coaching. London, England: Chartered Institute of Personnel and Development.

32.

Kaiser

R. B.

Craig

S. B.

(2005). Building a better mouse-trap: Item characteristics associated with rating discrepancies in 360-degree feedback. Consulting Psychology Journal: Practice and Research, 57, 235-245.

33.

Kimball

S. M.

(2002). Analysis of feedback, enabling conditions and fairness perceptions of teachers in three school districts with new standards-based evaluation systems. Journal of Personnel Evaluation in Education, 16, 241-268.

34.

Kluger

A. N.

DeNisi

(1996). The effects of feedback interventions on performance: A historical review, a meta-analysis, and a preliminary feedback intervention theory. Psychological bulletin, 119, 254-284.

35.

Lee

V. E.

Smith

J. B.

(1999). Social support and achievement for young adolescents in Chicago: The role of school academic press. American Educational Research Journal, 36, 907-945. doi:10.3102/00028312036004907

36.

Louis

K. S.

Leithwood

Wahlstrom

K. L.

Anderson

S. E.

(2010). Learning from leadership: Investigating the links to improved student learning. New York, NY: Wallace Foundation.

37.

McCauley

C. D.

Moxley

R. S.

(1996). Developmental 360: How feedback can make managers more effective. Career Development International, 1(3), 15-19.

38.

McDowall

(2008). Using feedback in coaching. In Passmore

(Ed.), Psychometrics in coaching: Using psychological and psychometric tools for development (pp. 26-44). London, England: Kogan Page.

39.

Miles

M. B.

Huberman

A. M.

(1994). Qualitative data analysis. Thousand Oaks, CA: Sage.

40.

Mone

E. M.

London

(2010). Employee engagement through effective performance management: A practical guide for managers. New York, NY: Routledge.

41.

Murphy

Hallinger

Heck

(2013). Leading via teacher evaluation: The case of the missing clothes? Educational Researcher, 42, 349-354.

42.

Murphy

Elliott

S. N.

Goldring

Porter

A. C.

(2007). Leadership for learning: A research-based model and taxonomy of behaviors. School Leadership and Management, 27, 179-201.

43.

Patton

(2001). Qualitative research and evaluation methods. Thousand Oaks, CA: Sage.

44.

Polikoff

M. S.

May

Porter

A. C.

Elliott

S. N.

Goldring

Murphy

(2009). An examination of differential item functioning on the Vanderbilt Assessment of Leadership in education. Journal of School Leadership, 19, 661-679.

45.

Porter

A. C.

Murphy

Goldring

Elliott

S. N.

Polikoff

M. S.

May

(2009). The VAL-ED technical manual. Nashville, TN: Vanderbilt University.

46.

Porter

A. C.

Polikoff

M. S.

Goldring

E. B.

Murphy

Elliott

S. N.

May

(2010a). Developing a psychometrically sound assessment of school leadership: The VAL-ED case study. Educational Administration Quarterly, 46, 135-173.

47.

Porter

A. C.

Polikoff

M. S.

Goldring

E. B.

Murphy

Elliott

S. N.

May

(2010b). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. Elementary School Journal, 111, 282-313.

48.

Price

H. E.

(2012). Principal–teacher interactions. Educational Administration Quarterly, 48, 39-85.

49.

Riemer

Bickman

(2011). Using program theory to link social psychology and program evaluation. In Mark

M. M.

Donaldson

S. I.

Campbell

(Eds.), Social psychology and evaluation (pp. 104-138). New York, NY: Guilford Press.

50.

Smither

J. W.

London

Reilly

R. R.

(2005). Does performance improve following multisource feedback? A theoretical model, meta-analysis, and review of empirical findings. Personnel Psychology, 58, 33-66.

51.

Sosik

J. J.

Megerian

L. E.

(1999). Understanding leader emotional intelligence and performance. Group & Organization Management, 24, 367-390.

52.

Supovitz

Sirinides

May

(2010). How principals and peers influence teaching and learning. Educational Administration Quarterly, 46, 31-56.

53.

Strauss

A. L.

Corbin

(1990). Basics of qualitative research: Grounded theory procedures and techniques (Vol. 1). Newbury Park, CA: Sage.

54.

Thomas

D. W.

Holdaway

E. A.

Ward

K. L.

(2000). Policies and practices involved in the evaluation of school principals. Journal of Personnel Evaluation in Education, 14, 215-240.

55.

Vohra

Singh

(2005). Mental traps to avoid while interpreting feedback: Insights from administering feedback to school principals. Human Resource Development Quarterly, 16, 139-147.

56.

Walker

A. G.

Smither

J. W.

Atwater

L. E.

Dominick

P. G.

Brett

J. F.

Reilly

R. R.

(2010). Personality and multisource feedback improvement: A longitudinal investigation. Journal of Behavioral and Applied Management, 11, 175-204.

57.

Yin

R. K.

(2009). Case study research: Design and methods (Vol. 5). Thousand Oaks, CA: Sage.