Efficacy of the Educational Leadership in the Spanish Context: The Perspective of Its Agents

Abstract

Our study aims to discover the degree of efficacy of the principal’s performance in relation to the learning objectives, curriculum, and results, and explore the relationship among the perceptions of the inspectors, teachers, and management teams about the principal’s efficacy in promoting students’ learning. The Vanderbilt Assessment of Leadership in Education (VAL-ED) questionnaire was applied. Contingency analyses were calculated. We concluded that the three sectors considered the principal’s work to be effective in actions related to the learning objectives, curriculum planning, and responsibility for the outcomes.

Keywords

efficacy educational leadership principal inspector teacher

Introduction

The renovation of the leadership area, with relevant theoretical orientations (Firestone & Riehl, 2005), has increased the need for its evaluation, and this is true in the Latin American context as well (Bolívar, 2015). Educational leadership has become a key factor in improving schools and students’ learning. It is a reality that does not go unnoticed. We require procedures and instruments that focus exclusively on the aspects of leadership that are critical in improving education and the functioning of schools. Research in recent decades has provided evidence about the types of leadership practices that influence students’ learning and well-being (Louis, 2015; Robinson, 2011).

School principals, when exercising educational leadership, have become the second most relevant factor in learning improvement (Leithwood, Louis, Anderson, & Wahlstrom, 2004). A certain consensus, increased by numerous investigations, shows that schools that have the capacity to improve are schools with principals who contribute, through their motivation, to helping the school to learn to develop, overcoming the challenges and difficulties encountered. Therefore, as Robinson (2006) proposes, “there is a need to redirect research on educational leadership so that it makes stronger links with curriculum, pedagogy, assessment and student learning and gives less emphasis to what I call ‘generic leadership’”(p. 63). The instrument we use focuses precisely on these dimensions.

Learning-centered leadership is focused, indirectly, on creating the conditions for optimal teaching and learning, and it can be evaluated by the improvement in the students’ achievement. In order to make a school more effective, contributing an added value to the student outcomes, it is important to concentrate on the curriculum, teaching, and assessment. The more the focus on these dimensions, and particularly on the conditions that make them possible, the greater the improvement will be in the school’s outcomes.

Thus, there is a need, as Fullan (2014) states, to “reposition the role of the principal as overall instructional leader so that it maximizes the learning of all teachers and in turn of all students” (p. 6). Therefore, it is relevant to evaluate to what degree the educational leadership of the principal increases the school’s performance levels.

The study we present here is part of a broader investigation financed by the Spanish Ministry of Economy and Competitiveness.¹ Specifically, our study aims to find out, on the one hand, the principals’ degree of efficacy in relation to the learning objectives, the curriculum (contents), and the academic results achieved in the school, according to the teachers, inspectors, and members of the management team (head of studies, secretary, and principal). These aspects correspond to three of the dimensions contemplated in the Vanderbilt Assessment of Leadership in Education (VAL-ED) questionnaire, which we have adapted and applied to the Spanish context. On the other hand, we explore the relationship between the position the agents hold (inspector, teacher, or management team) and their vision of the principal’s efficacy in terms of the school’s learning objectives, curriculum, and results.

Theoretical Framework

Effective School Principals

Leadership has a strategic position because of the ability to articulate diverse variables that, in isolation, would have little impact on the learning processes, but that together produce synergies that significantly increase this impact. Any improvement initiative depends on what happens in each school, and to a large degree, this depends, as a catalyzer, on the principal’s leadership. As Louis, Leithwood, Wahlstrom, Anderson, and Michlin (2010) state, “it is therefore difficult to imagine a focus for research with greater social justification than research about successful educational leadership” (p. 7).The reviews of the research (Day et al., 2009; Hallinger & Heck, 1998; Marzano, Waters, & McNulty, 2005) confirm that the effects of educational leadership on learning are greater in disadvantaged contexts or in schools with low performance levels.

Educational leadership also plays a relevant role in the teachers’ professional development. This impact on improving students’ learning is produced indirectly through its influence on the organizational and functional conditions of the teachers’ job in the school and the teaching quality. It can create the conditions and settings for good teaching and teacher learning through an exchange of good practices and other school policies that contribute to increasing the professional capital of the school (Hargreaves & Fullan, 2012). In addition, efficacious leaders use the resources strategically to prioritize the learning objectives, set goals and clear expectations, and ensure an organized environment that supports learning.

If principals have to perform multiple tasks related inevitably to administration and management, their core mission is education, and so they should focus all their efforts on it, and everything else should be directed toward improving the education offered in the school. Schools should guarantee the key competences to all their students, and the school leadership is there to make this possible, focusing their efforts on this goal.

Teachers’ quality can be strengthened, therefore, by the principal’s actions in this field (Day et al., 2010; Harris, 2014). It is important which school leadership practices create a context where teachers and, together, the whole school can work better, positively influencing the improvement in the students’ learning. The main practices the research has shown are the following: defining the vision, values, and direction; staff development; restructuring the organization: redesigning roles and responsibilities; and enhancing teaching and learning. Successful principals use the same basic leadership practices, but they “must be enacted in ways that are sensitively appropriate to the contexts in which leaders find themselves” (Day et al., 2011, p. 3)

We have accumulated knowledge about how school leaders can contribute to improving the education offered by their schools. School principals have to direct their actions toward redesigning the working contexts and professional relationships, producing settings where teams of teachers can learn together, and increasing the “social capital” of a school by increasing its professional capital (Hargreaves & Fullan, 2012). Moreover, in order to have teacher quality, a key factor in their effectiveness, it is necessary for them to have professional autonomy, commitment, and expertise (Day, 2017).

Leadership is practiced in context, producing a significant shift in the research from leadership styles or models to successful leadership practices. As Day et al. (2011) point out, “leadership success is heavily dependent on the qualities and skills through which leaders understand the underlying causes of the problems they encounter and respond to those problems in ways that are productive in context—not in general. Contexts, we argue, are unavoidable elements of the problems leaders need to solve if they are to improve their organizations” (p. 14).

School Principals in Spain: From a Bureaucratic Orientation to Educational Leadership

School principals in Spain have deep historical roots in a corporative model—where the principal is elected among and by the teachers—rather than being a professional figure (Viñao, 2004). For this reason, they have had few attributions allowing them to perform educational leadership in secondary schools. However, significant changes are beginning to occur in the way school management takes place in Spain. These changes are already reflected in new regulatory legislations in a transition from a bureaucratic model to a pedagogical orientation designed to improve learning and results in schools (Spanish Ministry of Education, 2007).

Spain and Portugal are the only European Union countries where the school principal is a teacher at the school (with at least 5 years teaching experience) chosen by the school council (teachers, parents, and students). At the end of the period as head (4 years, renewable), he or she returns to teaching (Ritacco & Bolívar, 2017). This system of being chosen by one’s colleagues (along with parents and students) has its origins in the end of the Franco dictatorship, when there was a demand for a “democratic running of the schools,” and where the organs of school management would be chosen by the school community. The participatory or democratic model de facto became corporative because the principal is overtly dependent on the colleagues who elected him or her (teachers are the majority on the school council). At the same time, she or he continues to be the representative of the education administration in charge of implementing and enforcing its regulations. This dual function is a continuous source of problems (Coronel & Fernández, 2002).

School management teams in Spain up until now have had little chance of exercising educational leadership, being limited in the majority of cases to administrative tasks within the existing organizational structures. A critical issue in the management and organization of schools is what the head can and cannot do to improve the instructional work of the teacher in the classroom and, consequently, the students’ learning. It is necessary to balance the principle of participation with the demands of professionalization, while ensuring the most appropriate professionals for the position of headmaster. In any case, the problem is not so much that the director is chosen by the teachers, but rather the way in which the organization where he or she works is structured. In this context, some VAL-ED items did not make complete sense to the respondents.

The administrative-bureaucratic model of school leadership, inherited in Spain, has serious deficits in stimulating the schools in late modernity. In the current conditions, a role limited to the bureaucratic management of the schools is becoming insufficient. The school’s main responsibility is to guarantee the educational success of all its students. However, it is important to consider what tasks and responsibilities principals in our schools should have and, based on them, promote the necessary changes in the organizational structure of the school’s new educational policy. In this regard, the current Education Act introduces, as a novelty, among the competencies of the principal, the responsibility “to exercise educational leadership.” Thus, we are in a transition from a bureaucratic model of administrative management to a pedagogical leadership, with all the problems usually found in periods of transition.

Therefore, it is important to evaluate the leadership practices in the Spanish context, where school principals have had a role more oriented toward management, with few educational leadership functions (Bolívar & Moreno, 2006). This makes the adaptation of the Spanish model especially costly and difficult because management teams do not usually have these functions, so that many items are foreign to the “school culture” established for principals, teachers, and inspectors. In Spain, the management team in state secondary schools is made up of the principal, the head of studies, and the secretary.

Vanderbilt Assessment of Leadership in Education

Within the “generally dreadful state of principal evaluation,” as Smith and Smith characterize it (2015, p. 3), or the ‘perfect storm’ of failure” that Reeves (2009, p. 1) describes, among the relevant efforts made to reshape the leadership evaluation landscape, we find the VAL-ED (Wallace Foundation, 2009). Goldring et al. (2008) analyzed 65 educational leadership evaluation tools, detecting serious deficiencies, where the dimensions of rigorous curriculum and quality instruction received little attention.

The VAL-ED is a questionnaire for the evaluation of the educational leadership capacity of the school principal (Porter et al., 2008). Developed by academics from the universities of Vanderbilt and Pennsylvania, its applications have obtained satisfactory evidence of validity and applicability (Goldring, Porter, Murphy, Elliot, & Cravens, 2009; Porter et al., 2010a). Our research team carried out its translation, adaptation, and validation in the Spanish context. The contribution of this adaptation has important consequences for two reasons. First, it is the first instrument to measure the principal’s efficacy in Spanish-speaking countries; and second, it allows comparisons to be made with the international reality.

Regarding its structure, the VAL-ED questionnaire contains 72 items, with a conceptual framework of two key dimensions of leadership practices: six core components and six key processes that define learning-centered leadership (Goldring et al., 2009). The six core components refer to different dimensions of educational leadership focused on learning, at both the literature and practical levels. They are noteworthy interventions that improve teachers’ learning opportunities and teaching practice with the students. Therefore, other leadership aspects are not included in the evaluation, such as values and knowledge, which, although important, are not part of the leadership behaviors focused on learning (Goldring et al., 2009; Murphy, Goldring, Cravens, Elliott, & Porter, 2011). These six core components are the following:

High standards for student learning: The extent to which leadership ensures that there are individual, team, and school goals for rigorous student academic and social learning

Rigorous curriculum: Ambitious academic content provided to all students in core academic subjects

Quality instruction: Effective instructional practices that maximize students’ academic and social learning

Culture of learning and professional behavior: Leadership ensures that there are integrated communities of professional practice at the service of student academic and social learning

Connections to external communities: Leading a school with high expectations and academic achievement for all students requires robust connections with the external community.

Performance accountability: There is individual and collective responsibility among the leadership, faculty, students, and community for achieving high performance standards for students’ academic and social learning

Each basic component is evaluated by focusing on the key processes of planning, implementing, supporting, advocating, communicating, and monitoring. These processes, as in any innovation process, refer to the way in which leadership, individually and collectively, influences organizations, in order to advance toward achieving the basic components. This framework is designed to evaluate the intersection between what the principals on leadership teams should do to improve students’ academic and social learning (basic component), and how to create these basic components (the key processes).

Of the six core components of the VAL-ED, in this article we focus on the analysis of the results of this educational trio: high standards for student learning, rigorous curriculum, and performance accountability. Many studies about leadership (Day et al., 2009; Leithwood & Louis, 2011; Robinson, Hohepa, & Lloyd, 2009) have identified the following as relevant practices: establishing goals at school, setting high learning standards, supervising the curriculum taught, and evaluating its effects on students’ achievement. Another aspect of this pedagogical tradition is that the learning objectives, contents, instructional strategies, and assessments should be aligned in a coherent way.

Progressively, the VAL-ED has been developed as a powerful instrument to evaluate the school principal’s educational leadership. It has also shown satisfactory validity and reliability (Murphy et al., 2011; Porter et al., 2010b). The VAL-ED was designed with the intention of providing a holistic, “360 degree” view, given that it includes the perceptions of the school management team, the faculty, and the inspectors or supervisors. It is referred to as a holistic evaluation where contributions are obtained from all angles, providing feedback for leaders on the essential components and processes of VAL-ED.

Focused on successful educational leadership practices of the principal, it is based on proposals (“background”) that leaders must be evaluated based on actions associated with students’ learning. Learning-centered leadership is ultimately measured by the students’ outcomes. It focuses, therefore, on learning, curriculum, teaching, and assessment (Goldring et al., 2009). The other organizational, administrative, or management dimensions are important as long as they help improve the student’s learning.

As an evaluation model, the VAL-ED is inspired by the understanding of leadership as the “process of influencing others to achieve mutually agreed upon purposes for the organization” (Goldring, Xiu, Murphy, Porter, & Elliott, 2015, p. 182). It focuses on behaviors associated with learning-centered leadership, measuring the efficacy of the principal’s interventions that have the most impact on the teachers’ activity and, indirectly, on students’ learning. The architecture for evaluating the principal has to take into account, as in the VAL-ED, that “the capabilities that are required for effective instructional leadership should inform the development as well as the evaluation of instructional leaders” (Smith & Smith, 2015, p. 23).

The Core Competencies and Key Processes are anchored by and aligned with the ISLLC 2008 Educational Leadership Policy Standards, one of the most important and recognized sets of standards, because they “represent consensual agreement across all the professional associations about the grounding for school administration,” and they “exert considerable influence on the shape and texture of the profession of school administration” (Murphy, 2015, p. 721). The basis for the conceptual framework consists of the actions and leadership practices that contribute an “added value” to the results achieved by the students. This model proposes that the evaluation of school leadership must include measures of the intersection between dimensions: The principals and management teams must achieve improvements in academic and social learning for all the students (the core components) and in the way in which these core components are created (the key processes).

Method

To develop the study, the descriptive method was used (Cohen & Manion, 1990), and for the data analysis, descriptive statistics and contingency analysis were applied. The information collection instrument was the questionnaire, which was administered during the academic year. It was preceded by a letter signed by the director of the research. Following the rigorous ethical protocol for research, the letter explained the study objectives, the benefits of participating, and the conditions.

The questionnaire was translated, adapted, and validated in the Spanish context. As Beaton, Bombardier, Guillemin, and Feraz (2000) recommend, in order for the instrument to conserve its properties of reliability and validity, a contextual adaptation is necessary. More than a faithful translation, adapting an instrument requires the consideration of cultural, idiomatic, linguistic, and contextual aspects (Hambleton & Zenisky, 2011). In our case, in addition, initially there was no “construct equivalence,” given that learning-centered leadership is very different, in practice, in the Anglo-Saxon and Spanish cultures. Therefore, a cross-cultural adaptation was carried out in order to guarantee a validity and reliability similar to those of the original instrument. Therefore, the phases suggested by the experts in this field were followed (Hambleton, Merenda, & Spielberger, 2005).

The VAL-ED was designed and developed to be both reliable (offering precise measurements) and valid (measuring the leadership behaviors that lead to increasing students’ performance) for use in primary and secondary schools (Porter et al., 2010b). It has had a broad psychometric development, and its validity has been shown. Cronbach’s alpha is .95 for both principals and teachers (Porter et al., 2010a). As shown in recent studies (Goldring et al., 2015), it is highly correlated, in a convergent way, with another measure of instructional leadership, the Principal Instructional Management Rating Scale (Hallinger & Wang, 2015), whereas it correlates weakly with more general measures of leadership.

In our adaptation to Spanish in secondary schools, the reliability of the questionnaire was calculated with Cronbach’s alpha coefficient, yielding a result of .978 and a Spearman–Brown coefficient of .957, so that the alpha for each of the two halves was .977 for the first half and .975 for the second. To determine the validity, we applied principal components. The KMO was .958, and Bartletts’ sphericity test was significant, χ²(136) = 1514.397; p < .001. The components obtained explained 61.517% of the variance, with the percentages for each factor being quite homogeneous. We can say that the six-dimension structure of the questionnaire was replicated because the items were grouped together in six factors. The cutoff point for assigning the items to the factors was 0.400.

Each of the items that make up the components has two response scales. First, the subjects have to point out the source of evidence on which they base their opinion about the efficacy, taking six options into account: Reports from others, Personal observations, School documents, School projects or activities, Other sources, and No evidence. Second, they have to rate the degree of efficacy with which the principals perform this behavior or practice at their school on a Likert-type scale that considers five degrees of performance: ineffective, minimally effective, satisfactorily effective, highly effective, extremely effective, and don’t know.

The study population was composed of Secondary Education teachers in public schools, the different members of the School management team, and the educational inspection. Given the complexity of accessing these sectors because of data protection and bureaucratic requirements of the different administrations, we decided to use accidental, quota (Kalton, 1983; Qian, 2010), or purposeful (McMillan & Schumacher, 2006) nonprobability sampling and sacrifice the representativeness of the national scope. Therefore, to access the sample, we contacted members of the Spanish Federation of the European Forum of Educational Administrators, the High School Principals’ Association of Andalusia, and the Federation of Associations of Principals of Public Schools, who provided us with access to the sample. We obtained 221 questionnaires out of the 850 that we distributed; 51 were from inspection, 99 from teachers, and 71 from management team members. Of them, 100% belonged to Secondary Schools, and 82% had more than 4 years of experience in the position.

Results: Discrepancy Matters

In order to use the information collected, and consistent with our research objectives, we applied the descriptive statistics that provided us with information about the efficacy of the school leadership. In addition, we used contingency analysis to examine the relationship between the position held and the view of the principal’s efficacy with regard to three components: the school’s learning objectives, curriculum, and results. We are particularly concerned with possible discrepancies between principals’ self-evaluations of their own efficacy and the perception that other sectors (inspectors and teachers) have of them. The ratings of others, particularly coworkers and subordinates, are very important in the principals’ evaluation because they explain much more than the self-evaluation does (Reeves, 2009). The discrepancy among the different sectors in the perception of the principals’ efficacy matters, as Sinnema, Robinson, Ludlow, and Pope (2015) remarked.

Regarding the descriptive practiced, the teachers, inspectors, and management team in our study considered the principal’s performance to be efficacious in setting learning standards. The processes of planning, implementing, supporting, advocating, communicating, and monitoring the school objectives, as the median indicated, had a score of about 3, showing efficacy. Regarding the school curriculum, these different processes are also considered effective (Mdn 3) by the teachers, the inspectors, and the management team. Its planning, accountability for the results, development, support, communication, and monitoring were also defined as effective (Mdn 3).

Further examining the contingency analysis, we applied it to the first component, which referred to the principal’s performance in establishing and communicating high standards for student learning. As Sun and Leithwood (2015, pp. 500-501) point out, “direction setting is a key leadership function” that, in some models, usually includes “framing the school’s goals and communicating the school’s goals.” In the VAL-ED, as in other questionnaires, “these direction-setting practices generally aim at the identification, development, and articulation of a shared vision that is appealing and inspiring to staff, achieving goal consensus among staff, motivating staff with challenging but achievable goals, communicating optimism about future goals, and giving staff an overall sense of purpose for their work” (Sun & Leithwood, 2015, p. 501).

In this component, high standards for student learning (Table 1), we observe that there are significant associations, depending on the type of factor, on 9 of its 12 items. The associations are present in all the processes of the questionnaire (planning, implementing, supporting, advocating, communicating, and monitoring), except inclusion. Regarding planning objectives, the inspectors, teachers, and management team in the study consider that the principal is effective (p = .000) when planning goals that the teachers will carry out in order to improve the students’ learning (Item 2), but they disagree (p = .003) about the principal’s efficacy in planning high learning standards for all students (Item 1). In this case, whereas the teachers and, especially, the management team find the principal’s leadership to be effective in this area, the inspectors mainly rate it as (42%) not very efficient.

Table 1.

Component: High Standards for Student Learning.

	Percentage					χ²	p
	1	2	3	4	5	χ²	p
1. Plans rigorous growth targets in learning for all students
Inspector	6.0	42.0	38.0	12.0	2.0	23.734	.003
Teacher	6.1	30.3	46.5	13.1	4.0
School management team	0.0	12.9	54.3	30.0	2.9
2. Plans targets of faculty performance that emphasize improvement in student learning
Inspector	7.8	33.3	41.2	13.7	3.9	33.238	.000
Teacher	5.2	27.8	49.5	12.4	5.2
School management team	0.0	10.3	45.6	42.6	1.5
3. Creates buy-in among faculty for actions required to promote high standards of learning
Inspector	6.1	36.7	40.8	16.3	0.0	19.684	.012
Teacher	8.3	32.3	38.5	14.6	6.2
School management team	4.5	14.9	41.8	35.8	3.0
4. Creates expectations that faculty maintain high standards for student learning
Inspector	0.0	50.0	38.0	10.0	2.0	31.681	.000
Teacher	16.5	25.3	40.7	13.2	4.4
School management team	2.9	20.6	48.5	26.5	1.5
6. Supports teachers in meeting school goals
Inspector	2.0	21.6	45.1	29.4	7.1	23.742	.003
Teacher	11.3	16.5	44.3	19.6	8.2
School management team	0.0	7.2	44.9	40.6	7.2
9. Communicates rigorous goals for student learning to faculty
Inspector	4.2	33.3	45.8	16.7	0.0	25.475	.001
Teacher	16.9	32.6	38.2	7.9	4.5
School management team	1.5	27.7	44.6	26.2	0.0
10. Communicates with families and the community about goals for rigorous student learning
Inspector	3.9	13.7	47.1	29.4	5.9	15.543	.049
Teacher	7.4	18.1	43.6	18.1	12.8
School management team	1.4	18.3	39.4	38.0	2.8
11. Monitors student learning against high standards of achievement
Inspector	8.2	32.7	38.8	20.4	0.0	28.127	.000
Teacher	14.3	24.2	47.3	8.8	5.5
School management team	4.3	11.4	41.4	34.3	8.6
12. Monitors disaggregated test results
Inspector	6.1	18.4	24.4	26.9	2.0	25.600	.001
Teacher	8.3	27.1	39.6	15.6	9.4
School management team	0.0	7.2	43.5	33.3	15.9

Regarding the development process, in our study, the inspectors, teachers, and, in a noteworthy way, the management team members think that the principal is efficient when establishing agreements among teachers to promote high learning standards (p = .012). However, half of the inspectors (50%) believe that the principals’ leadership is not very effective at creating high expectations in the teachers to maintain highly demanding standards for students’ learning (Item 4).

All the agents agree (p = .003), however, that the secondary school principal is effective when supporting teachers so that they identify with the school’s objectives (Item 6).

The actions undertaken by the principal in communicating the learning standards (Items 9 and 10) are also classified as effective by all the subjects involved in the study (p = .001 and p = .049, respectively). However, especially the inspectors in the study (45.8%) believe that the principal’s leadership is efficacious when he or she communicates high learning goals to the teachers (Item 9). In addition, the inspectors (47.1%) show greater agreement in that the principal is effective at informing the families and communities about the learning goals (Item 10). In this case, the ratings of the members of the management team are especially positive, as 38% also tend to classify them as quite efficacious.

Finally, all the agents consider the principals’ work effective in supervising the students’ learning to reach higher achievement levels (Item 11) (p = .000). However, the meticulous supervision of the results by the principal (Item 12) is considered quite efficient or efficient by the management team (76.8%) and the inspection (51.3%).

The second component is one of the key elements in the instructional core model by Elmore (City, Elmore, Fiarman, & Teitel, 2009). In this model, if we aim to increase the students’ learning and the teachers’ knowledge and skills and student engagement, we should increase the level of the contents. In this dimension, of the 12 items that make up the rigorous curriculum (Table 2) component, there are significant associations on 6. The associations are present in all the dimensions of the questionnaire (planning, implementing, supporting, advocating, communicating, and inclusion), except monitoring.

Table 2.

Component: Rigorous Curriculum (Content).

	Percentage					χ²	p
	1	2	3	4	5	χ²	p
13. Develops a rigorous curriculum for all students
Inspector	4.0	28.0	46.0	20.0	2.0	22.045	.005
Teacher	7.4	13.8	53.2	19.1	6.4
School management team	0.0	5.6	52.1	33.8	8.5
14. Plans access to rigorous curricula for students with special needs
Inspector	102	28.6	32.7	26.5	12.5	16.014	.042
Teacher	9.7	20.4	46.2	18.3	5.4
School management team	2.9	10.3	47.1	36.8	2.9
15. Creates rigorous sequences of learning experiences/courses
Inspector	10.9	45.7	28.3	13.0	2.2	16.313	.038
Teacher	13.8	32.2	41.4	8.0	4.6
School management team	8.8	17.6	50.0	20.6	2.9
18. Supports teachers to teach a curriculum consistent with state and national content standards
Inspector	3.9	17.6	54.9	19.6	3.9	22.664	.004
Teacher	8.3	16.7	37.5	26.0	11.5
School management team	0.0	5.6	49.3	40.8	4.2
19. Advocates a rigorous curriculum that honors the diversity of students and their families
Inspector	3.9	31.4	39.2	21.6	3.9	30.001	.000
Teacher	7.5	12	51.6	20.4	7.5
School management team	4.3	5.7	37.1	48.6	4.3
21. Discusses state curriculum frameworks
Inspector	6.4	38.3	38.3	12.8	4.3	25.731	.001
Teacher	15.6	33.3	40.0	6.7	4.4
School management team	10.1	11.6	49.3	27.5	1.4

Regarding curriculum planning (Items 13 and 14), the inspectors, teachers, and management team in the study coincide in considering that the school principal is efficacious when planning a curriculum for all the students (p = .005) and when planning access to a rigorous curriculum for students with special needs (p = .042). However, on this latter point, the inspectors’ opinions are not as consistent (32.7%) as those of the management teams (47.1%) and teachers (46.2%), who seem to agree more.

These differences between the group of inspectors and the other agents in the study are intensified in the dimension related to curriculum development. In this case, the faculty and, above all, the management team, think that the work carried out by the principal to create rigorous sequences that facilitate good learning (Item 15) is effective, whereas 45.7% of the inspectors rate it as not very effective. As occurs in the majority of European countries, for the inspectors, the function of “control has no connection to educational effectiveness” (Ehren, 2016, p. 25).

The inspectors’ opinions change when rating the support provided by the principal to the teachers in carrying out teaching consistent with the official curriculum (Item18). Regarding this point, all the subjects coincide in considering this support to be effective (p = .004).

Likewise, a very significant association (p = .000) is observed between the type of agent participating in the study and the efficacy conferred to secondary school principals in proposing a curriculum that addresses the diversity of students and their families (Item 19). In this case, the members of the management team who participated in the study consider the principals to be quite effective (48.6), whereas the faculty (51.6%) and the inspection think that they are only effective.

The opinions also differ when valuing the principals’ actions to guarantee a debate about adapting the official curriculum to specific contexts (Item 21). The management teams (49.3%), followed by the faculty, tend to classify them as effective, although the latter show a lower level of agreement (40%). However, the inspectors show divided opinions in this regard because 38.3% find them not very effective, whereas the same proportion think that they are effective, making their divergence on this aspect clear.

In the case of the third component, performance accountability (Table 3), in which the principals make the commitment to achieve high levels of students’ academic and social learning, we find that there are significant associations, depending on the type of agent, on 9 of the 12 items that make up this component. The significant associations are present in all the dimensions of the questionnaire (planning, development, support, communication, and monitoring), except inclusion.

Table 3.

Component: Performance Accountability.

	Percentage					χ²	p
	1	2	3	4	5	χ²	p
61. Develops a plan for individual and collective accountability among faculty for student learning
Inspector	8.3	43.8	33.3	12.5	2.1	26.379	.001
Teacher	13.3	23.3	52.2	7.8	3.3
School management team	4.8	12.9	53.2	27.4	1.6
63. Uses faculty input to create methods to hold faculty accountable
Inspector	2.0	33.3	47.1	15.7	2.0	37.498	.000
Teacher	7.4	13.7	48.4	21.1	9.5
School management team	0.0	5.8	43.5	46.4	4.3
64. Implements social and academic accountability equitably for all students
Inspector	4.3	32.6	43.5	15.2	4.3	31.516	.000
Teacher	9.7	9.7	55.9	16.1	8.6
School management team	1.5	3.0	59.7	29.9	6.0
65. Allocates time to evaluate student learning
Inspector	3.9	13.7	49.0	29.4	3.9	23.082	.003
Teacher	3.1	10.4	61.5	16.7	8.3
School management team	0.0	4.3	39.1	46.4	10.1
66. Allocates time to evaluate faculty for student learning
Inspector	22.9	22.9	35.4	16.7	2.1	18.475	.018
Teacher	12.6.	24.1	43.7	12.6	6.9
School management team	4.5	15.2	43.9	31.8	4.5
69. Discusses progress toward meeting school goals with parents
Inspector	0.0	33.3	45.8.	18.8	2.1	22.210	.005
Teacher	11.1	16.7	52.2	11.1	8.9
School management team	2.9	21.7	44.9	27.5	2.9
70. Communicates to faculty how accountability results will be used for school improvement
Inspector	0.0	46.0	34.0	18.0	2.0	43.011	.000
Teacher	11.1	11.1	48.9	18.9	10.0
School management team	6.2	7.7	52.3.	30.8	3.1
71. Analyzes the influence of faculty evaluations on the rigor of the curriculum
Inspector	10.9	26.1	41.3	21.7	0.0	25.143	.001
Teacher	13.2	15.4	54.9	6.6	9.9
School management team	9.0	10.4	44.8	31.3	4.5
72. Monitors the accuracy and appropriateness of data used for student accountability
Inspector	6.1	30.6	38.8	24.5	0.0	38.295	.000
Teacher	17.4	18.6	46.5	5.8	11.6
School management team	1.5	10.8	46.2	33.8	7.5

In planning, there is a significant association between Item 61 and the type of agent participating in our study (p = .001). The results confirm that half of the faculty (52.2%) and the management team (53.2%) agree that the principal is mainly effective in guaranteeing that the secondary school has a plan that promotes the teachers’ social and collective responsibility for the students learning, but 43.8% of the inspection members disagree and consider this planning to be fairly ineffective.

With regard to taking into account the teachers’ contribution to maintaining their accountability for the results (Item 63), the management team members tend to rate the actions carried out by the school principals as quite effective (46.4%), whereas teachers and inspectors mainly value them as only effective. In the case of Item 64, all the subjects agree that the principal is efficacious when developing social and academic accountability in an equitable way for all students (p = .000). Thus, the most satisfied agents are the members of the management teams (59.7%), followed by the faculty (55.9%) and, in last place, the inspectors (43.5%). The inspectors are usually more demanding about evidence based on the school outcomes (Scheerens & Ehren, 2015).

Regarding the support shown by the principal, the inspectors (49.0%) and, especially, the faculty (61.5%) agree that the principal is effective when dedicating time to evaluating the students’ learning, but the management teams show greater satisfaction with this aspect (46.4%). In this same dimension (support), the degree of agreement between the study agents converges when it comes to rating the principal’s efficacy in dedicating time to evaluating the teachers’ contribution to the students’ learning (Item 66). In this case, they all agree that it is efficacious (p = .018), especially the management team members (43.9%) and the faculty (43.7%) and, to a lesser degree, the inspectors (35.4%).

Something similar occurs when evaluating the way in which the principal informs the families about progress on the school goals (Item 69) because all the agents who participate in the study coincide in considering this work effective (p = .005). On this point, the management team, followed by the faculty and the inspectors, is more satisfied with the degree of efficacy achieved. This harmony among the subjects is broken (p = .000) when rating the principals’ efficacy in communicating to the faculty how the results obtained will be used for school improvement (Item 70). In this case, the management team (52.3%) and the faculty (48.9%) tend to find it basically effective, but the inspectors (46.0%), less satisfied with this aspect, classify it as not very effective.

Finally, regarding the monitoring carried out by the school principals, all the agents coincide in considering it effective (p = .001) when analyzing the influence of teacher evaluations on improving the curriculum (Item 71) and supervising the accuracy and suitability of the data used to improve the students’ results (Item 72). Nevertheless, in both cases, the inspectors agree less (41.3% and 38.8%, respectively) than the faculty and the management team about these aspects.

Discussion

If we focus on the significant relationships between the dimensions and items of the components of the VAL-ED presented in this article and the type of agents participating in this survey, we can confirm that, generally, the inspectors, faculty, and management teams in our study tend to consider the principals’ work to be effective.

However, if we further examine the component related to the learning objectives, we find that there is a great discrepancy between the inspectors and the faculty and, above all, the management team members, as they alone consider the principals to be not very effective at creating expectations in the teachers about maintaining high learning standards and planning rigorous goals for all students (planning and development tasks). The only aspect that the inspectors rate higher than the faculty and the management team is the efficacy of the principals’ actions when they inform the families and community about the learning goals.

The faculty, however, finds the principal less efficacious than the inspectors and the management team do at establishing agreements among the teachers to promote high learning standards, supporting them so that they identify with the school’s goals, informing the families and community about the learning goals, and communicating high learning goals to the faculty (development, support, and communication). According to Terek, Glušac, Nikolic, Tasic, and Gligorovic (2015), the communicative competence of the educational leader influences the teachers’ satisfaction because by offering support, improving the efficacy of the communication, and considering the teachers’ contribution, principals can contribute to improving the working conditions and satisfaction with the leadership.

In this research study, the management team members are especially satisfied with the principals’ efficacy, compared with the inspectors and teachers, in planning rigorous learning goals for all students, planning goals the teachers will carry out to improve learning, establishing agreements among the teachers to promote high learning standards, and supporting the teachers so that they identify with the school’s goals (planning, development, support).

As occurs in other contexts (Sinnema, et al., 2015), the principal’s self-evaluation is considerably superior to the evaluation by others. Thus, the members of the management team give a higher rating to the school principal’s actions related to the learning standards, whereas the inspectors question them more, except in the following four aspects: when establishing agreements among teachers to promote a high learning standards, supporting the faculty to help them identify with the school’s goals, communicating high learning goals to the faculty, and informing the families and community about the learning objectives.

If we focus on the curriculum, we find that the inspectors who disagree most, especially with the management team, consider that the principal is not very effective at creating rigorous sequences to facilitate good learning (development). As in other contexts, the perceptions of the principals do not usually coincide with those of the inspectors (Bitan, Haep, & Steins, 2015).

Inspectors also consider principals less effective than the faculty and management team do at planning a curriculum for all students, especially for students with special needs, and at proposing a curriculum to address the diversity of students and their families (planning and inclusion). The faculty, however, rate them as less efficient than the inspectors and management team do at discussing the official curricular adaptation (communication). The management team members, by contrast, consider that the principals are especially efficacious at actions directed toward planning the curriculum, supporting the teachers in providing an education coherent with the official curriculum, and proposing a curriculum that addresses students’ and families’ diversity (planning support and inclusion). As occurred with the objective components, the results confirm that the management team members are the most satisfied with the actions undertaken by the school principal regarding the curriculum, and the inspectors are the most dissatisfied with the level of efficacy reached. The faculty is not very satisfied with the principals’ efficacy in discussing the adaptation of the official curriculum to the context, an opinion that is shared by the inspectors.

The data obtained in the contingency analysis of Accountability for the results confirm the positions already highlighted in the two basic components of the questionnaire (objectives and curriculum): In our study, the agents who more highly rate the principal’s actions related to accountability for the results are the members of the management team, and those who express more doubts are the inspectors. The priority of the inspection systems is accountability, and the principals’ priority is improvement: “the requirements of accountability and improvement are in tension with each other” (Penzer, 2011, p. 14). This is the case for all the items that have a significant relationship with the type of agent collaborating in this investigation. We can state, thus, that the inspectors have doubts about the principal’s efficacy on 9 of the 12 aspects of accountability for results included in the University of Vanderbilt leadership questionnaire. This position of the educational inspection contravenes the considerations mentioned by Leithwood and Sun (2012), among others, and it emphasizes the influence of school leaders on the school’s results and the students’ performance. This critical view by inspectors is especially visible in the actions performed by the principal to create a school plan that promotes the teachers’ social and collective accountability for students’ learning, in the time established to evaluate the teachers’ contribution to the students’ learning, and in the way in which the principal communicates show that the results will be used for school improvement.

At this point, it should be pointed out that the inspection in Spain, unlike in other countries (Grek & Lindgren, 2015), has mainly had the function of controlling the reliable implementation of the regulations and standards of the Educational Administration, rather than advising about improvement (Ehren, 2016), at the same time that it supervises the school’s functioning, the curriculum, and the educational programs (Esteban-Frades, 2014). This external role of supervising curriculum development explains this critical attitude of dissatisfaction, compared with what internal agents in the school show (teachers, school management team). Thus, in this study, the results reveal that the teachers and members of the management team rate the principal more highly on accountability for the results, even though the latter consider it especially effective to take into account the teachers’ contribution to maintaining accountability for learning, developing social and academic accountability in an equitable way for all students, and taking time to evaluate students’ learning.

Conclusions

Based on the aims of this investigation and the data provided by our participants, we conclude the following:

The principals’ performance in relation to the learning objectives, the curriculum, and the results is considered efficacious. As in other evaluations, the “leaders are likely to rate their effectiveness more positively than their superiors, peers, or subordinates do” (Sinnema et al., 2015, p. 277).

There is an association between the position held (inspector, teacher, or management team) and the person’s vision of efficacy related to the learning objectives, the curriculum, and the results; the most critical agents when rating the efficacy of the actions carried out by the principal in these aspects are the inspectors, compared with the teachers and the management team.

The key processes of planning, implementing, supporting, advocating, and communicating are significantly associated with the learning objectives, the curriculum, and the learning results.

Finally, as limitations, we can point out that the VAL-ED measures the perceptions of the principal’s leadership efficacy, but, as the best studies on the topic recognize (Robinson, 2011; Smith & Smith, 2015), the evaluation of the school principal’s leadership has to be related to the results and the impact it has on improving the faculty’s teaching and increasing the students’ learning. Creating a favorable environment for this should be the principal’s main goal, and so the VAL-ED should be complemented by other variables that correlate these two dimensions (principal’s leadership and the impact on school improvement) in reality, and not only perceptively.

Another limitation was the low participation of the inspectors and principals. It is true that in our country the research on management teams and inspectors usually reveals this level of participation. Our study reflects the typical general pattern in the Spanish context.

One of the contributions of the study is the psychometric effort applied to the questionnaire, which obtained high reliability and validity and, therefore, can be used with guarantees of scientific rigor in the Spanish-speaking context.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Notes

ORCID iD

Purificación Pérez-García

Author Biographies

Purificación Pérez-García is a PhD and a senior lecturer in the School of Education, University of Granada, Spain. Her research interests are related to teacher education; period initial training; practicum; university teaching; professional ethics. She has published books and articles about this matters, and also, she has investigated about it.

Carmen López has a PhD in pedagogy and she is a senior lecturer at the University of Granada (Spain). She teaches at the Faculty of Science Education and her lines of research are: initial training, teachers’ professional development, practical training, intercultural education and university didactics.

Antonio Bolívar is a professor of curriculum in the Faculty of Education, University of Granada, Spain. His research interests are related to educational change; organizational and professional development; and educational leadership, about which he has published over 20 books and more than one hundred articles. He has carried out more than 20 financed research projects.

References

Beaton

D. E.

Bombardier

Guillemin

Feraz

M. B.

(2000). Guidelines for process of cross-cultural adaptation of self-report measures. Spine, 25, 3186-3191.

Bitan

Haep

Steins

(2015). School inspections still in dispute an exploratory study of school principals’ perceptions of school inspections. International Journal of Leadership in Education, 18, 418-439.

Bolívar

(2015). Assessing Pedagogical Leadership of Principals. A Review of the Frameworks and Instruments. Revista Iberoamericana de Evaluación Educativa, 8(2), 15-39.

Bolívar

Moreno

J. M.

(2006). Between transaction and transformation: The role of school principals as education leaders in Spain. Journal of Educational Change, 7(2), 19-31.

City

E. A.

Elmore

R. F.

Fiarman

S. E.

Teitel

(2009). Instructional rounds in education: A network approach to improving teaching and learning. Cambridge, MA: Harvard Education Press.

Cohen

Manion

(1990). Educational research methods. Madrid, Spain: La Muralla.

Coronel

J. M.

Fernández

M. L.

(2002). Is there a way out? A critical analysis of participation, leadership and management in Spanish schools. International Journal of Leadership in Education, 5(4), 303-322.

Day

(2017). School leadership as an influence on teacher quality. Profesorado. Revista de Currículum y Formación del Profesorado, 21, 139-157.

Day

Sammons

Hopkins

Harris

Leitwood

Kington

(2009). Impact of school leadership on pupil outcomes: Final report. University of Nottingham, England: UK Department for Children, Schools and Family Research.

10.

Day

Sammons

Hopkins

Harris

Leithwood

Brown

(2010). 10 strong claims about successful school leadership. Nottingham, England: National College for Leadership of Schools and Children’s Services.

11.

Day

Sammons

Leithwood

Hopkins

Brown

Ahtaridou

(2011). Successful school leadership: Linking with learning and achievement. Maidenhead, England: Open University Press.

12.

Ehren

(Ed). (2016). Methods and modalities of effective school inspections. Berlin, Germany: Springer.

13.

Esteban-Frades

(2014). Inspection in Education. History, thought and life. Oviedo, Spain: ADIDE de Castilla y León y KRK ediciones.

14.

Firestone

Riehl

(Eds). (2005). A new agenda: Directions for research on educational leadership. New York, NY: Teachers College Press.

15.

Fullan

(2014). The principal: Three keys to maximizing impact. San Francisco, CA: Jossey-Bass.

16.

Goldring

Cravens

Murphy

Elliot

S. N.

Carson

Porter

A. C.

(2008).The evaluation of principals: What and how do states and districts assess leadership? Paper presented at the annual meeting of American Educational Research Association, New York.

17.

Goldring

Porter

Murphy

Elliot

S. N.

Cravens

(2009). Assessing learning-centered leadership: Connections to research, professional standards and current practices. Leadership and Policy in Schools, 8(1), 1-36.

18.

Goldring

Xiu

Murphy

Porter

Elliott

(2015).The convergent and divergent validity of the Vanderbilt Assessment of Leadership in Education (VAL-ED): Instructional leadership and emotional intelligence. Journal of Educational Administration, 53, 177-196.

19.

Grek

Lindgren

(Eds). (2015).Governing by inspection. London, England: Routledge.

20.

Hallinger

Heck

R. H.

(1998). Exploring the principal contribution to school effectiveness: 1980-1995. School Effectiveness and School Improvement, 9, 157-191.

21.

Hallinger

Wang

W. C.

(2015). Assessing instructional leadership with the Principal Instructional Management Rating Scale. Dordrecht, Netherlands: Springer.

22.

Hambleton

R. K.

Merenda

P. F.

Spielberger

C. D.

(2005). Adapting educational and psychological tests for cross-cultural assessment. Mahwah, NJ: Lawrence Erlbaum.

23.

Hambleton

R. K.

Zenisky

A. L.

(2011). Translating and adapting tests for cross-cultural assessments. In D. Matsumoto Vijver

F. J.

(Eds.), Cross-cultural research methods in psychology (pp.46-74). New York, NY: Cambridge University Press.

24.

Hargreaves

Fullan

(2012). Professional capital: Transforming teaching in every school. New York, NY: Teachers College Press.

25.

Harris

(2014). Distributed leadership matters: Perspectives, practicalities, and potential. Thousand Oaks, CA: Corwin Press.

26.

Kalton

(1983). Introduction to survey sampling. Newbury Park, CA: Sage.

27.

Leithwood

Louis

K. S.

(2011). Linking leadership to student learning. San Francisco, CA: Jossey-Bass.

28.

Leithwood

Louis

K. S.

Anderson

Wahlstrom

(2004). Review of research: How leadership influences student learning. New York, NY: Wallace Foundation.

29.

Leithwood

Sun

(2012). The nature and effects of transformational school leadership: A meta-analytic review of unpublished research. Educational Administration Quarterly, 48, 387-423.

30.

Louis

K. S.

(2015). Linking leadership to learning: State, district and local effects. Studies in Educational Policy and Educational Philosophy, 1(3), 7-17.

31.

Louis

K. S.

Leithwood

Wahlstrom

K. L.

Anderson

S. A.

Michlin

(2010). Learning from leadership: Investigating the links to improved student learning. Final report of research findings. New York, NY: Wallace Foundation.

32.

Marzano

R. J.

Waters

McNulty

B. A.

(2005). School leadership that works: From research to results. Alexandria, VA: Association for Supervision and Curriculum Development.

33.

McMillan

J. H.

Schumacher

(2006). Research in education. Boston, MA: Pearson Education.

34.

Murphy

(2015). The empirical and moral foundations of the ISLLC Standards. Journal of Educational Administration, 53, 718-734.

35.

Murphy

J. F.

Goldring

E. B.

Cravens

X. C.

Elliott

S. N.

Porter

A. C.

(2011). The Vanderbilt Assessment of Leadership in Education: Measuring learning-centered leadership. Journal of East China Normal University, 29(1), 1-10.

36.

Penzer

(2011). School inspections: What happens next? Reading, MA: CfBT Education Trust.

37.

Porter

A. C.

Murphy

Goldring

E. B.

Elliot

S. N.

Polikoff

M. S.

May

. (2008). VAL-ED: The Vanderbilt Assessment of Leadership in Education. Technical manual. Nashville, TN: Discovery Education Assessments.

38.

Porter

A. C.

Polikoff

M. S.

Goldring

Murphy

Elliott

S. N.

May

. (2010a). Developing a psychometrically sound assessment of school leadership: The VAL-ED as case study. Educational Administrative Quarterly, 46, 135-173.

39.

Porter

A. C.

Polikoff

M. S.

Goldring

Murphy

Elliott

S. N.

May

. (2010b). Investigating the validity and reliability of the Vanderbilt Assessment of Leadership in Education. Elementary School Journal, 111, 282-313.

40.

Qian

(2010). Sampling. In Peterson

Baker

McGaw

(Eds.), International encyclopedia of education (pp. 390-395). Amsterdam, Netherlands: Elsevier.

41.

Reeves

(2009). Assessing educational leaders: Evaluating performance for improved individual and organizational results. Thousand Oaks, CA: Sage.

42.

Ritacco

Bolívar

(2016). Impacto del modelo español de dirección escolar en la identidad profesional de los líderes escolares [Impact of the Spanish model of school management on the professional identity of school leaders]. Archivos Analíticos de Políticas Educativas, 24(119). doi:10.14507/epaa.24.2512

43.

Robinson

(2006). Putting education back into educational leadership. Leading and Managing, 12, 62-75.

44.

Robinson

(2011). Student-centered leadership. San Francisco, CA: Jossey-Bass.

45.

Robinson

Hohepa

Lloyd

(2009). School leadership and student outcomes: Identifying what works and why: Best evidence synthesis iteration (BES). Wellington, New Zealand: Ministry of Education.

46.

Scheerens

Ehren

(2015). The evidence-base for school inspection frameworks. Journal of Educational, Cultural and Psychological Studies, 12, 43-76.

47.

Sinnema

Robinson

Ludlow

Pope

(2015). Discrepancy between teachers’ and principals’ perceptions of principal effectiveness. Educational Assessment, Evaluation and Accountability, 27, 275-301.

48.

Smith

J. R.

Smith

R. L.

(2015). Evaluating instructional leadership: Recognized practices for success. Thousand Oaks, CA: Corwin Press.

49.

Spanish Ministry of Education. (2007). School leadership in Spain: OECD country background report. Brussels, Spain: OECD Directorate for Education. Retrieved from http://www.oecd.org/dataoecd/33/51/38529289.pdf

50.

Sun

Leithwood

(2015). Direction-setting school leadership practices: A meta-analytical review of evidence about their influence. School Effectiveness and School Improvement, 26, 499-523.

51.

Terek

Glušac

Nikolic

Tasic

Gligorovic

(2015). The impact of leadership on the communication satisfaction of primary school teachers in Serbia. Educational Sciences: Theory and Practice, 15, 73-84.

52.

Viñao

(2004). School Management: a genealogical-cultural analysis. Educação (Porto Alegre), 27, 367-415.

53.

Wallace Foundation. (2009). Assessing the effectiveness of school leaders: New directions and new processes. New York, NY: Wallace Perspectives Occasional Series.