Abstract
This article reports research conducted recently into evaluation policy. The research comprises two parts: a questionnaire survey and qualitative interviews. Drawing from data collected in a survey of 65 curriculum leaders and interviews with 12 from the group, the article discusses how school evaluation policy functions to help make schools accountable to the community. This article reports findings from the study, which include the purposes, features and impacts of the policy as perceived by the curriculum leaders; and the strategies adopted by school teachers to deal with the evaluation procedures. It is found that the school evaluation policy has ‘dual characters’. The article then discusses the problems that may arise if the authorities fails to strike a balance between these dual characters, and instead puts too much emphasis on school effectiveness. It highlights the potential problems of excessive reliance on bureaucratic accountability and market-oriented accountability.
Keywords
Introduction
Recently, there has been a general increase in emphasis worldwide on evidence-based school evaluation. Evaluation in schools can serve various functions; accountability and decision making are the two typical ones (Hill, 1986). Another reason for this trend is greater decentralization and increasing autonomy for schools (Livingston and McCall, 2005; Janssens and van Amelsvoort, 2008). Efforts to increase accountability in public education have always sparked debates embedded in ideological differences about the purposes of public education. (Kuchapski, 1998: 188)
Accountability is not a mechanism but basically a principle that serves a purpose of a society. Hence, educational accountability is a political concept and it inevitably raises questions about who should have a stake in the educational process (Halstead, 1994; Macpherson, 1998). Its discussion often raises concerns about power and control. As a consequence, there are various approaches or models for establishing accountability (Kogan, 1986; Darling-Hammond, 1995; O’Day, 2002; Janssens and van Amelsvoort, 2008). 1 Table 1 summarizes the models/approaches most commonly discussed; they include the three models identified by Kogan (1986). The first is the public or state-control model (or bureaucratic accountability model), which is characterized by hierarchy and bureaucratized school and system organization. The second is the consumerist model, which may operate through market mechanisms. Finally, there is the professional model, which interprets teachers as autonomous professionals who can be directly accountable to parents. Each approach is based on a unique set of basic ideas and assumptions about schools and how they can be changed. They respond somewhat differently to the questions that define the meaning of accountability. 2
Approaches of accountability in education
Different social or institutional authorities may be practising or pushing for different kinds of accountability. Yet, in reality, many societies nowadays adopt a kind of test-based, high-stakes accountability system. Test scores are reported as providing evidence of differences in the quality of schooling. Some researchers criticize this kind of high-stakes accountability system as yielding a narrow policy focus, which has produced many harmful effects on the teaching profession (Maier, 2009). Good et al. (2010) make a noteworthy claim that over-reliance on standardized test results is problematic because it has devalued teachers, and displaced teachers’ professional autonomy in judging student learning.
Regardless of which model or approach of accountability a society adopts, schools undergo various models of school evaluation to ensure accountability and quality in education (Janssens and van Amelsvoort, 2008). In general, school evaluation includes internal evaluation and external evaluation. 3 These are described as ‘two sides of a coin’; both have their specific functions and are necessary elements for an effective system of school evaluation (Nevo, 2001; MacBeath and McGlynn, 2002). Schools in some countries experience mainly external inspection (for instance, England, Singapore and India); while schools in other countries adopt more liberal kinds of self-evaluation (for example, the Netherlands and Scotland) (Janssens and van Amelsvoort, 2008). Current research suggests that internal or self-evaluation has many positive effects on school development, including an improved evaluation understanding among teachers and the involvement of more stakeholders in school evaluation (Thornton et al., 2007; Bubb and Earley, 2008), thus promoting organizational learning (Robinson and Cousins, 2004; Hofman et al., 2009). Still, some authors consider internal evaluation to have a weakness in its limited credibility (Nevo, 2001). Indeed, external evaluation is often more objective and has more external credibility. However, many studies have discussed the negative impact of external evaluation. Sloan (2007) summarizes some negative impacts of external evaluation on teachers, curriculum and instruction, which include increasing anxiety and stress, resulting in teachers leaving the teaching profession, forcing teachers to deliver inferior instruction, and so on. Conversely, it was found that external evaluation had little impact in improving teachers’ instructional approaches (Ingram et al., 2004). Furthermore, Chapman (2001) demonstrates that teachers hold uncertain attitudes and views toward the influence of external evaluation. For instance, he found that only 50 per cent of teachers agreed that external inspection led to changes in classroom practice, and 51 per cent of them thought that the external reviewers did not get a realistic picture of their teaching. Other recent research has shown that external evaluation has little effect on student performance but has a severe negative impact on the personal identities of teachers (Hall and Noyes, 2009; Lugibuhl et al., 2009). The causes of these impacts are very much a cause of debate. Indeed, the culture of inspection is itself an important factor deciding how well external evaluation can function; Cullingford (1999) observed that a climate of trust between the inspectors and the school personnel is essential. Following this sort of advice, some educators stress the importance of maintaining a complementary and integrated relationship, or a critical friendship, between internal and external evaluation (Swaffield and MacBeath, 2005; Vanhoof and Van Petegem, 2007). To search for an ideal balance between internal and external evaluation, researchers propose various ways to combine internal and external evaluative processes (Nevo, 2001; Christie, Ross and Klein, 2004). Nevertheless, successfully matching the two processes is not simple and we are reminded to give careful thought to what the function of school evaluation is (Vanhoof and Van Petegem, 2007).
External evaluation is described by some authors as a judgmental and controlling process that helps to assess school effectiveness while internal evaluation functions as a developmental process that facilitates school improvement (Harris and Bennett, 2001; Livingston and McCall, 2005). The rationale behind the two traditions of school evaluation is different. Reynolds et al. (1996) compare school effectiveness and school improvement in a table (Table 2).
Comparison of ‘school effectiveness’ and ‘school improvement’
Source: Reynolds et al. (1996: 101 ) and Chapman (2005: 9).
In short, school effectiveness focuses specifically upon the outcomes of schooling and the characteristics of schools that make them effective, while school improvement is concerned mainly with the processes of schooling and the ways in which the quality of schooling can be enhanced (Harris and Bennett, 2001). School effectiveness stresses quantitative evidence of student advancement such as students’ academic performance in open examinations, value-added data, performance-indicators, and so forth (Darling-Hammond, 2004; Chapman, 2005). However, this tradition is severely criticized as having taken a limited or narrowly mechanistic view of education (Harris and Bennett, 2001; Good et al., 2010). Similarly, some researchers say that it may lead to problems of ‘standardization’: undermined teacher confidence and competence, amplified resistance to change and increased rates of stress and resignation (Hargreaves and Fink, 2006). Popham (2005) also argues that value-added methods, which emphasize students scores in state-mandated tests, sacrifice and displace instructional diagnosis as a way to cater for students’ learning needs. In contrast, the school improvement movement has argued that improvement and the capacity to improve come from within rather than beyond schools. Thus, school improvement aims to improve student outcomes by enhancing the school’s capacity to change (Stoll and Fink, 1996; MacBeath, 1999). For this to be achieved, it is argued that school self-evaluation must play a major role (Kyriakides and Campbell, 2004).
Since its return of sovereignty to Mainland China in 1997, Hong Kong has been experiencing consequent changes in education, and particularly in the purpose and model of schooling (Education Commission, 2000). Simultaneously, since 2003 policymakers have launched a system of internal and external evaluation. From the above analysis, it is quite clear that underlying the seemingly neutral technical processes of school evaluation policy, there often lie beliefs about how ‘education’ and society ‘ought’ to work. It is therefore essential to identify the ‘dominant normative theories, that is, statements of the desirable purposes and modes of accountability’ (Kogan, 1986: 17). What actually is the inherent or underlying stance of the current educational evaluation and accountability policy in Hong Kong? How will Hong Kong sustain a school evaluation policy in response to the quest for accountability? Which will it aim for: school effectiveness or school improvement? Could it strike a balance between internal and external evaluation? The study reported in this article identifies the nature and features of school evaluation in Hong Kong. It also indicates the kind of accountability the Educational Bureau (EDB) is promoting or practising. The discussion in the article will link the findings of the study to worldwide trends and to the significance of school evaluation policy for the purposes of education in a democracy. Hopefully, the article will illuminate practices concerning school-based curriculum evaluation, as well as issues regarding accountability and the school development movement.
School Evaluation Policy in Hong Kong
The history of the development of school evaluation policy in Hong Kong starts with its close ties to the UK. The concern about ‘quality’ in public services reached Hong Kong in the 1990s and soon began to make a strong impact on education (Comparative Education Policy Research Group, 1998). In fact the concern over school quality, school accountability and effectiveness was also a result of the implementation of 9-year compulsory education in 1978 (Tsang, 1997; Wong, 1997). Subsequently, the Education Commission put forward a number of recommendations in its report No. 7 to improve school accountability and performance towards provision of ‘quality school education’ in Hong Kong (Education Commission, 1997). This orientation was presented clearly in the report when it listed the key components of a ‘quality culture’: Setting clear and commonly accepted goals for school education and having these goals clearly understood by all players in the school system. Translating the goals into achievable, observable and measurable quality indicators for self-evaluation and external assessment. Allowing school management greatly autonomy in general administration, finance and personnel matters but at the same time requiring a higher degree of accountability for school performance. (Education Commission, 1997: 4–5)
Along with the report, the Education Department introduced a quality assurance framework (Figure 1). In this framework, Quality Assurance Inspection (QAI) served as an external quality assurance mechanism while schools were required to conduct school self-evaluation (SSE) as an internal quality assurance process. The QAI continued until 2003 when the EDB developed a school development and accountability framework (SDA), which comprised SSE and External School Review (ESR). Through SSE, schools could evaluate their own performance, improve themselves continuously and develop themselves into quality schools. Following a school’s SSE, the EDB would conduct External School Review (ESR) of public sector schools to validate their SSE. In this way, ESR would play a complementary role to SSE. So ESR was an external evaluation of the internal evaluation (SSE) carried out in schools. The Quality Assurance Division (2009) claims that SSE and ESR aim to achieve quality school education through school improvement and accountability.
Quality assurance framework
In another official document, the EDB affirms that the ultimate aim of SSE and ESR is the enhancement of school development and improvement of students’ performance in learning (Quality Assurance Division, 2008: 1).
To support SSE and ESR, a framework of performance indicators (PIs) was prepared to enable stakeholders to assess school performance (Figure 2). Schools are required to conduct their holistic review (SSE) with reference to the PI framework. The ESR procedure also uses this framework to validate how schools perform and self-evaluate. Both procedures seek to review school development and effectiveness. The framework of PIs originally comprised four domains that contained altogether 29 PIs (Quality Assurance Division, 2002). The four domains are management and organization, learning and teaching, student support and school ethos, and student performance.
Performance indicators framework
To facilitate the SSE process, the EDB has developed key performance measures (KPM) and measurement tools such as Standard Stakeholder Survey questionnaires (see the Appendix for detailed summary of the various tools for SSE). Schools are expected to use these instruments to assess their own performance.
Normally, a school will undergo one cycle of SSE and ESR every 3 or 4 years. During the actual implementation of ESR, an ‘ESR team’ (comprising EDB officials, and one external reviewer) visits a school for up to five days (known as the ‘ESR week’). Through ESR, the EDB hopes to achieve the following: to validate the school self-assessment (SSA) report presented by the school; to examine the school’s self-evaluation tools and processes. (Quality Assurance Division, 2007)
In preparation for SSE and ESR, a school needs to prepare documents such as a school self assessment report, a report card of key performance measures, and the results of a ‘stakeholder survey’ (Quality Assurance Division, 2007) for scrutiny by the ESR team. In 2008, this requirement was revised and schools are now required to provide only the latest two school development plans, annual school plans and school reports (Quality Assurance Division, 2009).
During the ESR week, the ESR team will scrutinize other school documents. It will shadow students, observe lessons, conduct focus group meetings with school staff, and meet parents. In theory the ESR team is said to operate in ways that ensure the external inspection process is independent, responsive, fair and open. All procedures are available to school staff and school boards. Discussion and communication between schools and the officials is to be encouraged as much as possible. A formal ESR report will be prepared and given to the school. The school is required to pass the ESR findings on to stakeholders such as the school management committee, teachers and parents (Quality Assurance Division, 2009).
On the whole, in Hong Kong, the SSE and ESR function together as an accountability framework, and simultaneously as an improvement cycle of the school. How do the ‘two sides of a coin’ (MacBeath and McGlynn, 2002) work together to enhance the internal and external accountability of schools? How open or closed is this framework in helping schools to maintain professional autonomy while making themselves accountable to the community? The present article may provide some insights to these questions from the perspectives and experiences of school curriculum leaders in Hong Kong.
Research Purpose and Methodology
This article was prepared after most schools in Hong Kong had gone through one round of SSE and ESR since the launching of the system in 2003/4. The present study was conducted at this point in order to capture the experience and opinions that school practitioners had acquired from this first cycle of SSE and ESR. This research was intended to investigate what curriculum leaders think of the school evaluation framework and its implementation in schools. It draws on a simple questionnaire survey administered to a random sample of curriculum leaders, as well as the qualitative data obtained from in-depth individual interviews with 12 of the curriculum leaders from the sample.
The research questions were: How do the curriculum leaders perceive the aims, procedures and requirements of the school evaluation policy? How have the curriculum leaders and their schools gone through the process of SSE and ESR? How would the curriculum leaders evaluate the policy?
In Hong Kong, school-based evaluation officially starts in the primary education section. There were altogether 557 primary schools in Hong Kong in 2008/9 (Census and Statistics Department, 2010). This study invited all the participants of a course, namely the ‘Training Programs for Primary School Curriculum Leaders 2008/2009’ commissioned by the Education Bureau, to participate in a questionnaire survey. The total number of key informants therefore included 65 curriculum leaders whose ranking was Primary School Master (Curriculum Development), or PSMCD in abbreviation. 4 These curriculum leaders are experienced primary school teachers in Hong Kong and have been leading the development of school curriculum for years. They were required to attend the training programme when newly appointed as PSMCDs. In Hong Kong, PSMCDs are teachers who are also responsible for leading and coordinating their colleagues in planning, improving and implementing curriculum. They are responsible for leading school teachers to plan and cope with the requirement of SSE and ESR. Their experience and perspectives are essential to this study.
To investigate the PSMCDs’ views on issues relating to school evaluation policy in Hong Kong, a simple questionnaire survey was conducted. Practical considerations led the researcher to invite a relatively small sample to participate in the survey. This survey, however, provides an indication of the ‘commonality’ and ‘variability’ of individual responses and perspectives to issues relating to SSE and ESR (Krathwohl, 2009: 568). The researcher revised, and then finalized the questionnaire after conducting a pilot test with a small sample of the target group.
Interviewing was another major research method used to follow up the opinions collected by the questionnaire survey in this study. Interviewing is an effective method of probing the subjective feelings of key participants (Weiss, 1994). It could help supplement the more objective quantitative data collected from the survey. Among the 65 curriculum leaders who returned questionnaires, 12 accepted the researcher’s request for an interview. These 12 PSMCDs were invited particularly because their schools were recently inspected by ESR. The researcher thought their experiences from this school evaluation exercise were therefore most fresh and relevant. Semi-structured interviews, each lasting around 1 hour, were conducted with each of these interviewees.
The analysis of official research reports as well as literature influenced the design of the research instruments—the questionnaire and the interview guide.
5
The researcher thereafter constructed the questionnaire to investigate how the PSMCDs understood and perceived the policy, and how they evaluated the design and impact of SSE/ESR. For the interview, one key focus was to probe the perspectives of these school leaders about the policy with regard to its influence on schools, curriculum, teachers and their work as school leaders. Major interview questions included the following: What do you think the purpose(s) of SSE and ESR are? How would you comment on the policy? How did SSE and ESR affect the school, the teachers and you? How did the school and you respond to the school evaluation policy?
Simple statistical analysis was undertaken to analyse the quantitative data collected from the survey. Responses to the questionnaire were tabulated and then compared by using percentages. To analyse qualitative data collected by interviews, ‘inductive analysis’ was employed (Patton, 2002). This is an analytical strategy to further analyse and interpret the interview data. The researcher identified and discovered patterns, themes and categories that emerged from the data. Results from quantitative and qualitative data were then triangulated, compared and contrasted. Final findings were then selected for further discussion.
Findings
The research consisted of two parts—a short questionnaire survey, followed by in-depth interviews. Quantitative and qualitative data thus collected could supplement or triangulate with each other. Also, the interviews functioned to investigate questions that emerged from the survey.
Survey
The survey was conducted in 2008/9. At the end of the last session of the PSMCD course, curriculum leaders were invited to answer a quick questionnaire. Altogether 65 returns were received. The survey data was then subjected to simple descriptive statistical analysis.
This section of the article presents the findings of this brief questionnaire survey under the following major themes. Curriculum leaders’: understanding of the purposes of SSE and ESR; views about the effectiveness of the school evaluation policy in improving local education; evaluation of the influence of the school evaluation policy to leadership (particularly for this group of informants who are curriculum leaders in schools); perception of the positive and negative impacts of the school evaluation policy on school improvement, curriculum and teaching, and so on; evaluation of the effect of various SSE and ESR components/instruments, including performance indicators, observation of teaching, and so on.
For easy reading and comprehension, most of the data analysis in this section will be presented alongside tables.
The survey aimed to discover what curriculum leaders thought was the EDB’s purpose in pushing forward SSE and ESR in schools (Table 3). Fifty-six out of 65 (86.2 per cent) of responsees went for ‘oversees the operation of the school, simultaneously scrutinizes the progress of school-based curriculum reform’. Forty-one (63.1 per cent) took the purpose as to ‘lure schools to attain school self-improvement as well as systematic school-based management’. Of all responses 19 speculated ‘“survival of the fittest”—to align with policy regarding school closure or resource allocation’ (29.2 per cent) as the purpose. Only 14 chose ‘help(ing) schools to verify the result of self-evaluation’ (21.5 per cent), which is the formal aim claimed by officials (Quality Assurance Division, 2008). This implies that most curriculum leaders view SSE/ESR as a way for the EDB to oversee school effectiveness; or as a tactic to prompts schools to strive for self-improvement.
Curriculum developers’ view of the purpose of SSE/ESR
When asked about which was the most effective EDB policy for pushing forward education reform, most curriculum leaders chose SSE and ESR (28 out of 65, 43.1 per cent). The school evaluation policy clearly out-rated other options (Table 4). This suggests that this official school evaluation policy functions, though indirectly, to put into effect educational reform agendas (see also Author, 2006).
Curriculum developers’ comment on SSE/ESR as a means to facilitate leadership role in schools
Furthermore, the curriculum leaders agreed almost unanimously that the school evaluation policy helped the development of the school and its curriculum. For instance, 62 curriculum leaders out of 65 answered that SSE and ESR affected their school’s overall curriculum planning ‘to a large extent’ (37/65; 56.9 per cent); or ‘to a certain extent’ (25/65; 38.5 per cent) (Table 5). Findings from the survey also show that many schools take close reference to the ESR framework to reform their administrative structure, allocation of human resources, curriculum and teaching. Of the 65 curriculum leaders 51 (78.5 per cent) replied that their schools had reorganized their administrative structure according to the PI framework. In follow-up interviews (see analysis on ‘interview’ below), some curriculum leaders described the ways that their schools had re-organized their administrative structure (for example, allocating teaching staff to four school development sub-groups which correspond to the four domains of the PI framework: management and organization, learning and teaching, student support and school ethos, and student performance).
Curriculum developers’ evaluation of the influence of SSE/ESR on their overall school curriculum planning
Moreover, 48 out of 65 (73.8 per cent) curriculum leaders affirm that SSE/ESR was an effective means that helped them to perform their leadership role in their schools (Table 6). Findings from the interview later will also show that the curriculum leaders find this evaluation policy useful in helping them to push teachers to change their teaching approach.
Curriculum developers’ election of the most effective official policy in improving local education
Curriculum leaders were also asked to identify SSE/ESR components that were most effective in inducing school development. Out of the 65 participants, 30 and 25 selected ‘classroom observation’ (46.2 per cent) and ‘performance indicators’ (38.5 per cent) respectively; followed by 19 who chose ‘key performance measurement’ (29.2 per cent). None accorded other tools, such as SVAIS, APASO, and so on, as being most effective to bring about change in school development. Interview data had similar findings to this—curriculum leaders consider practically that observation of classroom teaching and performance indicators are the most essential elements of the evaluation exercise. From analysis of interviews below, the curriculum leaders showed how they led teachers to face these challenges (Table 7).
Curriculum developers’ identification of the most effective SSE/ESR components/tools
When asked to choose one option which could best describe the advantages that were brought by this school evaluation policy to their schools, 55 out of 65 (84.6 per cent) curriculum leaders picked ‘help schools to understand own weaknesses, strengths and identify needs for improvement’; followed by ‘promote implementation of curriculum reform proposals’ (28/65; 43.1 per cent), ‘enhancement of teachers’ professional competence’ (16/65; 24.6 per cent) and ‘improving teaching approach’ (14/65; 21.5 per cent) (Table 8).
Curriculum developers’ views of the advantages brought on by SSE/ESR to schools
Curriculum developers’ view of the deficiency of SSE/ESR
Finally, the curriculum leaders were asked to identify an option that could best illustrate the deficiencies in the school evaluation policy. The most frequent choice was ‘the procedures put stress too heavily on paper work and documentation’ (36/65; 55.4 per cent) and ‘the policy has not taken enough account of the situation of individual schools’ (31/65; 47.7 per cent). These two viewpoints were raised for discussion in interviews. Eleven curriculum leaders chose ‘the professional knowledge of the ESR reviewers is inadequate’ (11/65; 16.9 per cent). Obviously, the curriculum leaders were dissatisfied with the additional workload of documentation, which was presumed to be valueless but essential for the ‘survival’ of their schools. The analysis of data from interviews will further elaborate on this point.
Interviews
Interviews with the 12 curriculum leaders followed analysis of the survey. The schools where these curriculum leaders work had recently undergone SSE and ESR. This research procedure helped the researcher to probe the participants’ perspectives and perceptions about SSE and ESR. Qualitative analysis started after the collection of interviewing data. During the process of identifying, coding and categorizing, patterns (recurring regularities) emerged from the data. These patterns were represented by three broad themes, which include curriculum leaders’ (1) perception of the purpose of SSE and ESR in Hong Kong; (2) attitudes and strategies towards SSE and ESR, and (3) perception of the negative impacts of the school evaluation policy. Each of the themes was characterized by several categories and/or sub-categories (Table 10). The following sections analyse these findings in greater detail.
Content analysis of interview data in the research
Source: Patton (2002).
Purposes of SSE and ESR: Ensuring Accountability and Effectiveness
This is the first prominent theme that emerged from the interview data. The majority of the 12 curriculum leaders thought that SSE and ESR was a measure imposed by the government to scrutinize the effectiveness of school-based management. This reinforces the survey findings. The curriculum leaders believed that the school evaluation mechanism was specially designed to help schools attain accountability to the public. To some of them, SSE/ESR was a rational public policy: It’s a strategy for the EDB to oversee how well schools perform, for the sake of accountability. (CD9: 3–8)5
It’s a policy used by the EDB to examine the effectiveness of schools in Hong Kong. (CD10: 11–12) I think the key purpose of ESR is to guarantee effectiveness and accountability of schools to the society. (CD 9: 15–16)
Purposes of SSE and ESR: Enhancing School Improvement
Some curriculum leaders implied that the policy was able to bring about school improvement. They think that this evaluation policy has pushed schools to plan and evaluate school curriculum in a systematic way. Some of them believe that the policy induces the professional development of schools and teachers. Judging with positive thinking, ESR is not bad because it can make schools become clearer about their focuses of development; as well as their SWOT
6
; more systematic and accountable to stakeholders. (CD4: 23–27) ESR makes us more aware (of EDB instruction), responsive and proactive in looking for evidence to demonstrate effective curriculum construction, as well as teaching effectiveness. (CD10: 12–15)
Attitudes and Strategies towards SSE and ESR
This is another interesting theme that emerges from the interview findings. The curriculum leaders commonly held a cautious, defensive attitude towards SSE and ESR. To help their schools to muddle through the mandatory evaluation mechanism, the curriculum leaders sought out ‘the most powerful ESR components’ and certain ‘coping strategies’.
The curriculum leaders mentioned that some ESR components were found most ‘powerful’. The items that were brought up by most curriculum leaders included (1) observation of classroom teaching and (2) the ESR report. Some other colleagues talked about APASO, TSA 7 and best quality assignment. These ESR components were described as ‘powerful’ in the sense that they were perceived by teachers/schools as threatening, sometimes terrifying; or they were felt to be directly influencing the destiny of a teacher’s self or the school. This finding coincides partially with that from the questionnaire survey (when the curriculum leaders were asked to identify the most effective ESR elements).
(a) Observation of classroom teaching—An ESR team will observe at least one lesson conducted by each teacher in a school. The curriculum leaders found that this was powerful enough to push forward change in teachers’ teaching. There is one ESR requirement that teaching of every teacher will be observed during an ESR visit. This scares many teachers. They fear that the lesson they demonstrate in front of those ESR officials may be accorded a low grade. Although the ESR team states that they will not under any circumstances mention teachers’ names to the principal, teachers are still very worried. This leads them to change their teaching approach in their classroom. (CD8: 56–60)
(b) ESR reports—Evidence shows that many believe that ESR reports have a direct effect on the destiny of a school. The curriculum leaders affirmed that the ESR report often became a determining force for schools to plan their school development framework: The whole school was very concerned about the ESR report (issued by the ESR team after the ESR). The ESR report is just like a school’s ‘report card’. All parents and the public will focus on the comments made by the ESR team. If the ESR report of a school is positive, the school will be able to attract more students. And this directly lessens the danger of school closure. (CD7: 48–52) We’ll take close reference from results shown in APASO and other KPMs, last round’s ESR comments, etc. to develop our school curriculum. (CD11:64–67)
The curriculum leaders mostly believe that a school should base its strategic planning on comments in the ESR report. Before making changes, the curriculum leader together with the senior school administration would carefully study the officials’ criticisms and suggestions for improvement. We (our school) made close reference to ESR report when planning our three-year school plan. For example, our last ESR said that our teachers were weak in questioning strategy. For this reason, developing teachers’ questioning skills became one focus in our next three-year plan; we organized a teacher development day and invited an external expert to provide in-school professional training to teachers. (CD5: 75–80)
All the curriculum leaders confessed that they used certain coping strategies to deal with the demands of ESR. These included: forming ‘study groups’; planning and acting to meet standards laid down by the PIs; preparing teachers to cope with classroom observation; much more writing and filing; and learning the competence to ‘speak’ and ‘present the school performance’ beautifully.
(1) Forming ‘study groups’ to have in depth understanding of the evaluation guidelines. The curriculum leaders said that their schools often formed teachers into ‘study groups’ to read the ESR guidelines carefully for generating strategies to cope with ESR. We would form into different study groups, each taking turns to study ESR domains (like curriculum and teaching, or assessment of student learning). Action plans would be prepared and submitted for review by the principal. (CD1: 200–206)
CD6’s elaboration was most explicit: We adopt the principle of ‘cooperative learning’, divide labour among four administrative groups. We follow EDB’s evaluation framework to name the four groups, i.e., … I found this strategy pretty effective. Each group is responsible for planning and evaluating the yearly School Development Plan. (CD6, 131–137)
(2) Plan and act to meet standards laid by the PIs. Some curriculum leaders said that they would carefully consider ways to work to the standards stated in the performance indicators (Quality Assurance Division, 2008). The PIs are taken by the curriculum leaders as official standards for schools in Hong Kong to strive for. Hence, they would study the PIs carefully; sometimes they would invite teachers in the school to study (the PIs) together: As the curriculum leader, I need to be the one who is most familiar with the criteria of school assessment. I should have a thorough understanding of all 23 PIs; then share with colleagues. We then brainstorm ways to achieve the standards stipulated in the PIs. (CD4: 54–57) I invite colleagues to join various ‘think-tanks’ to study carefully the meaning of the PIs. When colleagues propose an idea, I summarize and submit a proposal for the approval of the principal. This seems an effective way for our school to work out ways of reaching official standards. (CD5: 64–66)
(3) Prepare teachers to cope with classroom observation. Findings from both the survey and the interviews showed that many school teachers and curriculum leaders were aware that ‘observation of teaching’ was most important to reflect school effectiveness during ESR. For this reason, they had devised different strategies to ensure that better teaching performance would be shown to the ESR reviewers. For instance, some schools teamed up teachers to study the criteria that the ESR reviewers would employ to assess teachers’ teaching. We therefore form into groups to study carefully ESR’s assessment criteria of classroom teaching. We find that the observation record form for teaching is very useful. We therefore carefully look into the assessment items on the record form, brainstorm ideas and experiences in putting those requirement into effect. (CD3: 43–47)
The next thing is to improve teaching methods. Therefore, I often remind my colleagues (teachers) to ‘polish’ their teaching skills, train up the routine in classrooms, prepare their students to work in groups, etc. All these preparations could help them (teachers) to get good grades when the ESR team arrives to observe their teaching in classrooms. (CD10: 77–80)
Besides, most schools would conduct peer observation (of teaching), which is a useful way for teachers to view others’ teaching and to learn from each other how to improve. Some of these schools invited experts to hold workshops or a professional development day/retreat to give teachers a chance to refresh their pedagogical knowledge. One school employed an external organization, which claims to provide professional service to improve the quality of school education, to provide classroom observation for every teacher before the ESR team arrived. The company observed each teacher’s work, analysed possible problems, and provided expert suggestions to improve the school’s classroom teaching.
(4) Much more writing and filing. Many commented that SSE and ESR was mostly ‘documentation work’. They said that they had to prepare files of documentary evidence to show their performance in the four dimensions. One school claimed that they had prepared 30 or more files for the EDB officials to check (CD2). One of them said that the most important coping strategy was to write good school plans and evaluation reports: Well, dealing with ESR is not a difficult job for me. I’m used to writing and preparing documentation. My principal said that having clear, systematic and efficient documentation is the foremost strategy to make a good impression on the ESR team. (CD2: 126–127)
(5) The competence to ‘speak’ and ‘present the school performance’ beautifully. More interestingly, one curriculum leader said that the most effective tool for coping with the EDB’s ESR is ‘a clever mouth’. She claimed that ‘if you could present in a clever way when the officials interviewed you, your school would surely pass ESR with excellent grade!’ (CD4: 226–228). When asked how, she said: You should not say anything negative about your school. Yet you should not be too proud and say that your school is perfect. The way you present is to establish with facts and evidence the current achievements of your school. Equally important is to tell the officials that your school is aware that there are still aspects that need improvement and your school has a specific plan to deal with them in the next stage … (CD4: 230–237)
Besides, some curriculum leaders appreciated that ESR signifies the EDB’s stance that schools should not shrink from changes and initiatives. They thought that their SSE report should demonstrate that the school was progressive and willing to adopt innovations and improvements. We have to prove that we closely follow educational trends and needs. It is very beneficial to demonstrate how energetic and forward-looking our school is. Say for example, our school is currently promoting the rationale of ‘inclusive education’. In our SSE school plan, we specify this as a key task for development. During the last round of quality supervision, we emphasized our adoption of the policy of ‘using Putonghua to teach and learn Chinese subject’. (CD2: 96–105)
Negative Impacts of the SSE and ESR
The curriculum leaders observe that SSE and ESR had certain negative impacts to schools and teachers. They discussed ways that the evaluation policy had exerted influence on matters, such as teacher workload and stress, competition between schools, the academic-oriented tradition in Hong Kong, and neglecting the individual differences inside schools.
Increasing teacher workload and teacher stress. Many curriculum leaders said that the school evaluation policy increased teachers’ workload. As in the findings from some research elsewhere, teachers felt increased stress because of the SSE/ESR in Hong Kong. To prepare for the SSE and ESR, it’s inevitable that much workload is added to teachers. Teachers need to hold lots of meetings, prepare more paper work for recording meetings, or curriculum planning, or evaluation, etc. (CD7: 76–78) In my school, many teachers are tense and nervous about ESR. Most of them regarded ESR as threatening, with an exceptional fear of observation of teaching. Many of my colleagues did spend many days and nights to prepare a special lesson for this. (CD8: 20–24)
Intensified competition between schools. In recent years, schools in Hong Kong have had to compete for students. The curriculum leaders said that ESR had intensified the competition between schools. This is because most schools have the ESR report uploaded to the school’s website and disseminated to the public. This becomes a way for them to publicize the school’s achievements. The remarks made by the following two curriculum leaders were striking: Our school principal said that we needed to have something special in order to out-perform other schools. He often reminded me that ‘see, other schools have initiated this and that; we shouldn’t be lagging behind!’ It’s a matter of fact that schools nearby in the region need to compete with each other. And that’s the reason why we are often busily preparing for new initiatives. (CD2: 66–72) We publicized our new 3-year plan to parents during parents’ night, and some teachers were sent to distribute pamphlets in the shopping arcade near our school. We highlighted our ESR result in these pamphlets. That proved to be effective to attract more students! (CD5: 48–50) Failure to ‘outperform’ others (other schools) renders a school redundant and vulnerable, so a school will adopt strategies proven useful to get high scores from ESR. (CD12: 71–72)
Achievement-oriented tradition persists. Hong Kong is always criticized as being result-oriented; this tradition seems unshakable. Though the EDB has often stated that they would like to alleviate this problem, the school evaluation policy still is found putting pressure on schools to get results. Besides, curriculum leaders found they could not ignore the power of the ‘examination league table’. Their school principal cares very much where their school is on the somewhat informal ‘league table’ between schools in the district. This is directly related to the schools’ achievement and ranking in the community, which affects whether parents would choose to enrol their children in the school. Indeed, the EDB has its own ‘league table’, which is shown only to the school principal. The official league table is based heavily on TSA results. The principal will ask me and other leaders to follow up if our school does not get a favourable place on the league table. (CD2, 70–75) You couldn’t imagine how parents make this happen. From the ESR report issued to the school, parents can find comments the officials made about our school. ESR report is like a ‘report card’ issued to a school. Parents also equate the achievement of a school to its capability to send students to eminent secondary schools. (CD2: 60–68)
In actual fact, ESR expects to see how a school responds to its place in the league table, TSA result is one key component of the SSE report. We need to analyse the student achievement shown on TSA, then identify the strengths and weaknesses for drafting our school improvement plan. The ESR officials will then review how our school performs data analysis, self evaluation and planning according to TSA result. (CD2: 146–150)
Hence, it is logical for teachers and curriculum leaders to conclude that the government still puts great stress on the academic achievement of students. However assertive the officials sound when they promote the notion of ‘all-round education’ to the public, the curriculum leaders find that the hidden agenda is still result-oriented. Though it is not stated in the documents, everybody knows that the EDB is expecting to see school students get good academic results. Sometimes the reviewer will ask, ‘How does your school plan to deal with the comparatively low TSA results of students?’(CD8: 113–117) ESR reinforces the tradition of over-emphasis on ‘academic achievement’ cos’ you’ll see quantitative outcomes always seems to be more objective and convincing, in the eyes of the public! (CD11: 99–103)
Neglecting individual differences inside schools. As mentioned in analysis above, the curriculum leaders responded that ‘the policy has not taken enough account of the situation of individual schools’ (31/65; 47.7 per cent). This view was followed up during interviewing. More than one curriculum leader said that the officials appeared to pay little heed of individual differences, but expected schools to enable all students to attain equally good academic results; for instance, one PSMCD said: it is quite disappointing to find that the ESR team seemed to overlook the issue of diversity in classrooms in their comments on our school performance. They criticized that our academic result in some cases is too unsatisfactory; but they disregard the fact that our school has admitted many students who have special needs (such as cases of mild mentally retarded, or other physically handicapped). Many of our students are from families of low SES. How incoherent the officials sound in implementing educational policy! (CD8: 105–112)
When asked if the SVAIS (school value-added information system) as a tool helps in this respect, many of the CDs replied honestly that they barely used this tool for self-evaluation. They explained that this was because they had little knowledge about SVAIS, nor did they possess the skills to use it. They also revealed that the officials rarely asked for data about SVAIS during the ESR.
Summary
The qualitative and quantitative data collected from this research show how curriculum leaders in Hong Kong perceive and comment on the present school evaluation policy. Their key ideas or opinions about SSE and ESR include the following. The policy is an official, top-down policy to secure effectiveness and accountability of school education to the local community. One key focus of the official policy rests on improving teaching and learning in the classroom. The pair of school evaluation policies help the development of the school, its curriculum and the professional development of schools and teachers. Their experience with the policy: (a) have developed a number of ‘coping strategies’ to deal with the policy; (b) some ESR components, which were found comparatively most powerful, were tackled by some of these ‘coping strategies’. Shortfalls or potential threat of the policy include: (a) the procedures stress too heavily on paper work and documentation; this increases teachers’ workload and pressure; (b) some teachers got a clear official message during the ESR process: that the EDB was very concerned about students’ performance, in terms of test achievement; (c) the policy has not taken enough account of the situations and problems of individual schools, but uses the same standards to evaluate schools; all in all, students’ academic performance is a premier criteria; (d) competition between schools increased as a result: schools made efforts to achieve better ESR results, and to ‘climb’ the ‘examination league tables’ (for example, league table of TSA).
Discussion
This article found that the current school evaluation policy in Hong Kong presents pairs of features. These pairs of features can be described as dual characters of the school evaluation policy (SSE and ESR): it comprises both internal evaluation and external evaluation; it aims for school improvement and school effectiveness; it subjects schools to bureaucratic accountability plus market-oriented accountability.
The pairs of features relate to the current discourse and research agenda of school evaluation. The features are contrary in character, but in most cases they complement each other. Each of the pairs has particular characteristics and different influence on the nature and operation of the school evaluation policy. In some countries, the school evaluation policy demonstrates only one feature in the pairs—for instance, some European states have a school evaluation policy aimed at school self-improvement. Nonetheless contrasting features are found coexisting in the official policy of school evaluation in Hong Kong.
SSE and ESR—Internal and External Evaluation
This research shows how schools in Hong Kong are implementing a combination of internal and external modes of evaluation. SSE functions as the school’s internal evaluation process while ESR functions as the external evaluation through which officials scrutinize school effectiveness. This research confirms that the SSE and ESR act as a quality control mechanism, which guarantees ‘standards’ and ‘benchmarks’ are met by schools in Hong Kong. The standards or benchmarks are represented by a framework of PIs. There are three predominant models of self-evaluation and external evaluation in use in different administrations, including parallel, sequential and cooperative (Swaffield and MacBeath, 2005: 240). The SSE and ESR in Hong Kong follow the sequential model, with ESR (the external evaluation) following SSE (self-school evaluation).
Curriculum leaders confirmed in this study that SSE in Hong Kong is a mandated process that represents a verification phase within a QA system. This process may help to monitor and govern schools to ensure their public accountability. However, international research indicates that this is a kind of ‘narrowly defined’ SSE (Janssens and van Amelsvoort, 2008: 16). If ‘broadly defined’, SSE is seen as a self-initiated, development process contributing to improved teacher and student learning (Livingston and McCall, 2005). To many researchers, an ideal SSE aims to establish a climate or culture in which there is a shared belief that school improvement is the responsibility and expectation of every member in schools (Elmore, 2005).
In principle, internal evaluation and external evaluation are regarded as two interrelated areas of one strategy for school reform. They work as a pair to enhance school improvement and accountability. This study shows that Hong Kong, as most other countries in the world, has not struck the ideal balance between internal evaluation (SSE) and external evaluation (ESR) (Janssens and van Amelsvoort, 2008). The curriculum leaders demonstrated that SSE is considered by schools to be a prelude for ESR. Schools were alert to the fact that good preparation for SSE would help a school to achieve good result from their ESR. ESR was taken as the ultimate procedure when the inspectors would make judgments about the school. As a consequence, the professional value of SSE is diluted by ESR. Probably for this reason, the curriculum leaders focused much of their discussion on experience and opinions about ESR during interviews. Moreover, the present study also identified constraining factors that are said to be inhibiting the implementation of ideal SSE (MacBeath, 2006). Such factors include: workload; fear and ambiguity about the purpose of self-evaluation; the fear of colleagues seeing weakness (for example, poor teaching) as being harmful to the survival of the school, and so on.
Indeed, researchers have found that some tensions usually arise from an attempt to combine the two forms in one way or another (Janssens and van Amelsvoort, 2008). It is said that when the stakes are high in external evaluation, ‘schools might feel threatened and turn to strategic or opportunistic behaviour, such as withholding data, distorting data, manipulating pupils’ school careers, etc.’ (Janssens and van Amelsvoort, 2008: 17). This was generally the case in Hong Kong when the government first launched its QAI and then the SSE and ESR (Author, 2008). Quite disappointingly, a similar situation was still found in this study. The research shows that some curriculum leaders and their school teachers felt worried about the result they would get from ESR and hence they were compelled to adopt certain preventive strategies to counter the challenge of external review—digging out the most critical/powerful ESR components, and then developing a handful of coping strategies, such as thorough documentation, drilling classroom teaching, and so on. The conclusion can be drawn that the curriculum leaders found the school evaluation policy to be a dangerous game for high stakes. For teachers in Hong Kong, ESR is still a high-stakes exercise, through which many teachers experience a high degree of anxiety. This perception is at odds with the friendly stance by the EDB, which states that ‘the ultimate aim is enhancement of school development and improvement of students’ performance in learning. The school-led self evaluation is complemented and supported by external school review (ESR)’ (Quality Assurance Division, 2008: 1).
For School Effectiveness or for School Improvement?
School evaluation can aim at school effectiveness and/or school improvement (Reynolds et al., 1996; Harris and Bennett, 2001; Chapman, 2005) (Table 2). In this research, the curriculum leaders opined that they found the pair of imposed policies displayed features of both traditions. Superficially it seems that the evaluation policy merges school effectiveness and school improvement well. However, with a closer look at the findings, it can be seen that the pair of policies, as depicted from the leaders’ opinion, was actually highly management-focused in its approach to promoting quality education. In response to the policy, curriculum leaders and teachers spent more time to complete additional management-related work. When compared to Reynolds et al.’s (1996) description, the policy was found to be partial to school effectiveness. It was: focusing on schools and their organization; this explains why most schools re-organize their administrative/academic structures by the four formal domains; data-driven; emphasizing outcomes (school yearly outcome, and so on); mainly quantitative in nature (KPMs, APASO, SHS); more concerned with schools at a point in time (the ESR week; the 3-year plan and outcome of the school); more concerned with change in student outcomes (TSA, general academic result of the school); concerned with which schools were more effective, and which schools were not.
On the contrary, the policy still fails to demonstrate the following features, which are important in leading schools to improve: it targets schools but has not concentrated enough on individual teachers; it focuses on school organization rather than school processes; it pays little concern to the journey of school improvement; but pays heed to the outcome; it pays less attention to the process of change: how could schools become effective. (After Reynolds et al., 1996)
Currently, in many countries, merging school effectiveness and school improvement is recommended (Reynolds et al.,1996; Harris and Bennett, 2001). Still, it is said that a good self-evaluation process is itself a school development process (Reynolds et al., 1996). Hopkins et al. (1994: 3) consider school improvement to be: A distinct approach to educational change that enhances student outcomes as well as strengthening the school’s capacity for managing change. In this sense school improvement is about raising student achievement through focusing on the teaching and learning processes and the conditions that support it. It is about strategies for improving the school’s capacity for providing quality education in times of change, rather than blindly accepting the edicts of centralized policies and striving to implement these directives uncritically.
Indeed, the EDB in Hong Kong seems a long way from actualizing the vision of an improvement-oriented school evaluation policy described in its own documents (Quality Assurance Division, 2008, 2009) that ‘ESR will continue to be improvement-oriented … in line with the spirit of embedding SSE in ongoing school practice and encouraging schools to take greater responsibility for continuous development’ (Quality Assurance, 2009, ‘Foreword’– p.1).
Which Kind of Accountability?
The present research shows the school evaluation policy in Hong Kong embraces in practice bureaucratic accountability and consumerist (market-oriented) accountability.
Bureaucratic Accountability
In bureaucratic accountability a state or a city concentrates the authority and power of educational decision making in the central government. Its main characteristic is that of a managerial hierarchy (Kogan, 1986). This is ‘built on the idea that government policy rests on a chain of authority from super-ordinate units (the EDB, followed by school heads) to subordinate ones (school curriculum leaders, followed by teachers) (Cibulka, 1991: 183). Schools as ‘collective entities’ are ‘accountable to the higher levels of the educational system’ (Fuhrman and Elmore, 2004: 22). It is a ‘control model of accountability’ in which a power coercive strategy changes teachers’ behaviour in conformity with an externally imposed contract (agenda)’ (Fuhrman and Elmore, 2004: 37). Findings from this research imply that the education system in Hong Kong has especially focused on attempts to manage schooling through standardized SSE and ESR evaluation procedures, performance indicator framework, documentation, and so on. From curriculum leaders’ sharing of the reaction of school teachers toward the said policy (working to meet the standard of PIs, drilling for lesson observation, over-emphasis on preparing documents, learning to speak beautifully, and so on), it can be found that the EDB has applied, tactfully, a control model that shapes teachers’ behaviour in conformity with the EDB’s agenda. Obviously, the ESR and SSE serve as official management-based measures for promoting quality education (Mok and Welch, 2002). Clear evidence from the study shows that teachers have to spend more time in preparing mission statements, school plans/reports and management-related work instead of devoting more time to students and quality teaching.
In the SSE and ESR framework, the PIs are used for reference to evaluate school performance under four domains of school work. They are benchmarks for measuring schools’ performance. Although the EDB in Hong Kong often stresses that schools need not rate themselves against the PI framework, the PIs have inevitably turned out to be important criteria for schools. Rather than carrying out their duties according to professional expertise and experience in the best interest of students, teachers and school leaders revealed that they had tried efforts (such as re-structuring the internal administrative organization according to the PI framework) to meet the ‘indicators’. This problem is similarly met by many schools in the USA since the 1980s when many raised the concern that performance-based accountability narrows the purpose of schooling and ‘demeans and trivializes the work of educators’ (Elmore, 2005: 277).
Moreover, the performance indicators become effective tools to ‘provide decision makers and users of the education system with information on … the way the system operates, the effectiveness of the service provided, … and to render these systems more transparent’ (Henry et al., 2001: 91). Besides, this allows officials to mandate curriculum change (Author, 2008). The end effect may be stronger control of school education by the government. Indeed, using performance data to induce change by local school officials is the logic of bureaucratic efficiency (Cibulka, 1991).
In fact, this study shows that curriculum leaders and teachers in Hong Kong are quite used to working in a hierarchical, bureaucratic relationship with the EDB. The study shows that curriculum leaders were well prepared to comply with official requirements. In the first place, they sorted out the ‘most powerful weapon of ESR’; then they worked out different strategies to cope with ESR—preparing documents for scrutiny of the officials; forming ‘study groups’; knowing the way to say ‘clever words’; getting trust from the officials that the school has been progressing (but not delaying responding to the call of reform); and so on. Strategies similar with these are described as ‘intended’ or ‘unintended’ strategic behaviour used by schools to cope with school inspections (Wolf and Janssens, 2007). Wolf and Janssens (2007) found that schools the UK, the Netherlands, the USA also adopted different kinds of ‘intended or unintended strategic behaviour’ (gaming, window dressing, and so on) with an intention to be assessed more favourably by the assessor during inspections. Compared with these countries, certain tactics shown by schools in Hong Kong to cope with external reviews, such as forming ‘study groups’, ‘speaking beautifully’, were distinctive.
By and large, the present study showed that schools in Hong Kong treated ESR as little different from external inspection; all such proactive or reactive coping strategies enabled the schools to ‘pass’ the ESR with good grades. Nevertheless, this subjected schools and the local profession to ‘technical rationality’ (Schön, 1983). Moving further away from the ideal of school autonomy and liberty, the will of officials dominates the decision making of schools.
The Consumerist Model/Market-oriented Accountability
Besides the monitoring carried out by the EDB, this article found that the school evaluation policy is increasingly being governed by market forces. The evidence also implies that this tendency is indirectly fostered and reinforced by the EDB. So school evaluation policy functions with the perspective of the consumerist model or, as some call it, market-oriented accountability. In the consumerist model, the mechanism of the free market acts as the primary means of enforcing educational accountability (Kogan, 1986; Halstead, 1994; Leithwood and Earl, 2000). This model is based on the belief that ‘if schools no longer have a guaranteed clientele they will have an incentive to compete, which in turn will push up educational standards’ (Halstead, 1994: 151). In fact, this approach to accountability is also prominent elsewhere in the world. For example, versions of it are evident in several European countries, Canada, the USA, New Zealand, Australia, and parts of Asia (Leithwood and Earl, 2000).
In the present study, it was found that schools in Hong Kong compete with each other by seeking better ESR results and excellent student performance (in terms of test results); as well as a higher ranking than other schools in the community, and so forth. All these have become important targets for schools. In fact, student enrolment is the key motive for this competition. This explains why curriculum leaders mentioned that school survival rested on parental support. As a consequence, schools and the entire enterprise strive to win the market game.
Combining Bureaucratic and Consumerist Accountability
So schools were led to be involved in local systems of competition through SSE and ESR. However behind the screen, it is the government that steers the direction of this competition. As one curriculum leader said, ‘I see that the EDB uses ESR to align with its policy implementation—when the officials wish to push forward PMI policy, 8 they will pay much attention to schools which have put this into their school plan. And many parents ask if our school has also adopted this initiative’ (CD 8: 89–92).
Obviously the government understands this market-oriented context. By way of public propaganda, such as using media and ESR reports, the EDB promotes its preferences and priorities. This allows officials to mould the values of parents in local educational settings. Conversely, parents can access the SSE plans and ESR reports online; they compare the performance of different schools, as shown by examination results, the quantity as well as quality of extra-curricular activities, the number of teachers possessing post-graduate degrees, and so on. One may observe from this tricky cause-and-effect relationship how the Hong Kong government steers educational development; as well shaping the society’s ideology (Eisner, 2001; Marsh, 2009). Hence, this study reflects how schools are led to undergo both bureaucratic and consumerist accountability through the imposed school evaluation policy.
Conclusion
This article finds the current school evaluation policy in Hong Kong to exhibit a dual character. The SSE and ESR are: for external as well as internal evaluation of schools; for school improvement and for school effectiveness; for both bureaucratic accountability plus market-oriented accountability.
It is found that the evaluation policy tends toward school effectiveness. The perception and the reaction of school leaders and teachers toward the policy diverge to a certain extent from officials’ intention to encourage school-based improvement. It is recommended that the government in Hong Kong should gradually do away with the practice of employing effectiveness- or managerial-based policies to deal with schools (‘Author’ and ‘partner’, 2006). Rather, efforts should be made to realize a real improvement-oriented policy. This should enable stakeholders in a school to acquire knowledge of what to improve, how to improve and to develop motivation to engage in improvements. The process is then really developmental and lets stakeholders have a sense of ownership of the problems. The primary aim of an ideal self-evaluation process is to establish a climate, or a culture, in which there is a shared belief that everyone can make a difference and that school development is the right and responsibility of every single member of the educational community (MacBeath and McGlynn, 2002).
Another threat and shortcoming occurs with the last double-character pair—making schools combine bureaucratic with market-oriented accountability. This combination forms what is described as ‘a new alliance and new power bloc’—a phenomenon that has been formed contemporarily and has gained increasing influence in education elsewhere in the world. This new alliance combines the ideology of neoliberalism and neoconservatism (Apple, 2000, 2006). In brief, neoliberal marketized policy turns education over to the market through voucher and choice plans. Neoliberal policies subject schools to the discipline of market competition (namely, the concept of quasi-market solutions in education). In this case, markets exert substantial pressure on school development. Policies being proposed under the neoconservative position are mandatory curricula, state-wide testing, standard-based or performance-based, calling for raising standards and so forth (Apple, 2006: 39). Regrettably Hong Kong’s evaluation policy seems a partial concession to this ‘new alliance’.
In many countries, neoliberal visions of quasi-markets are accompanied by neoconservative pressure to regulate content and behaviour through such things as national curricula, standards and systems of assessment (Apple, 2000, 2006). Unfortunately, this is detrimental to an ideal development of local education as well as the democratization of the society (Whitty et al., 1998; Apple, 2000, 2006). The pair of tendencies (neoliberalism and neoconservatism) would ‘actually oddly reinforce each other and help cement conservative educational positions into our daily lives’ (Apple, 2000: 85). More time and energy will be spent on maintaining or enhancing a public image of a good school and less time and energy on pedagogic and curricular substance. Schools become more similar, and more committed to benchmarks, a standard and traditional (often monocultural) curriculum, in the examination-oriented tradition. Increasingly centralized control of an education service is likely to lead to the process of deprofessionalization (Nixon and Rudduck, 1994) and the disempowering of teachers. Hence, educators, policymakers, lawmakers and politicians, in addition to teachers and parents, all have social responsibility to keep school development from excessive marketization and bureaucratic control.
Soon, the EDB in Hong Kong is going to launch the second round of SSE and ESR. 9 The present researcher would recommend that officials review the school evaluation policy first. A professional perspective is suggested for officials engaged in evaluating schools (Leithwood and Earl, 2000; Fuhrman and Elmore, 2004). In actual fact, some countries, such as Italy, Germany and Switzerland, are becoming more decentralized (MacBeath, 2006) and some others, such as Belgium, Denmark and the Netherlands, have developed a more open framework of SSE to provide abundant freedom to schools and teachers (Janssens and van Amelsvoort, 2008). By trusting schools to be professionally accountable to society, they can give teachers and immediate leaders greater opportunities to develop skills, exercise judgment and take greater control over their work. Empowering the professionalism of schools, teachers and school leaders may lead to the emancipation of parents and students, and public concern for deconstructing and reconstructing the meaning of ‘schooling’ and ‘education’. This is critical to preventing dominant social groups from exercising ‘hegemonic processes’ so as to legitimate particular areas of knowledge in their society (Giroux, 1989). Furthermore, an evaluation which is committed to a professional rather than a bureaucratic conception of teaching and schooling can play a role in a democratic society (Nevo, 1997). This is especially important for a city like Hong Kong political and social structure of which experiences constant evolution. However, despite its promise, reliance on professional accountability alone is not proven to be a panacea for all problems. It is found that the overall development of professionalism in a society’s education is essential for the efficacy of this alternative approach to evaluation (O’Day, 2002). Obviously, this recommendation to the policy response in Hong Kong is tentative, looking forward to further empirical studies to prove its efficacy.
The political order and citizenship discourse in Hong Kong have been shown to be affected by the nature of political transition (the changeover of sovereignty after 1997). It is found that the social reality nowadays reflects a faithful demand or aspiration to full democracy (Hung, 2004). Indeed, Hong Kong’s people have the reasons (and many of them did 10 ) to ask for democracy and citizenship development in public policy. The Basic Law of HKSAR (Constitutional Affairs Bureau, 2004) has suggested the city will continue to enjoy all kinds of freedom and choice. 11 The Aims of Education issued by the EDB include democracy and citizenship as the key objectives of Hong Kong education (Education Commission, 2000). 12 Hitherto, many in the society are actively involved in the ongoing struggle for universal suffrage (for example, The Standard, 27 June 2003). In contrast, many officials and policymakers pay close attention to the stance taken by the People’s Republic of China (PRC) toward the democratization of Hong Kong. 13 Evidence shows that Hong Kong is in some danger of losing autonomy and political liberty: 14 some say that education and the school curriculum are a frontier to be protected. Public discussion of what and how school evaluation should continue is therefore essential.
It is said that the kind of accountability system and school evaluation model is rooted in particular economic, political and social contexts (Swaffield and MacBeath, 2005; Janssens and van Amelsvoort, 2008); it is equally correct that a public accountability policy has a role in ‘sustaining the moral economy’ of a community (Macpherson, 1996). This implies that a community needs a public accountability policy that can give expression to reconciling its plural principles, beliefs, values and interests. Hong Kong is facing tremendous challenges posed by a globalized economy. Reunification with China and democratization have changed the way Hong Kong people think and live. Policy makers are advised to take into account of the ‘evolving culture and mindframe to these changes (Education Commission, 2000: 3, 2.1)’ when formulating policies that really help to realize the rhetoric of creating ‘greater autonomy and capacities in administration (education) and teaching (Education Commission, 2000: 41, 6.32)’ in the community.
Appendix
The EDB in Hong Kong has developed the following evaluation tools and data in support of schools’ self-evaluation work (SSE).
KPM (Key Performance Measures)
A data system developed on the basis of the framework of performance indicators (PIs). It assists school to self-evaluate and report school performance.
SVAIS (School Value-Added Information System)
The SVAIS provides Hong Kong schools with confidential information on the extent to which they add value to the academic progress of their students. Schools are recommended to use SVAIS as tool for school self-evaluation. With SVAIS, schools can compile value-added reports and analyse the value-added patterns. With reference to other available information (for example, students’ performance in the HKAT, internal assessment results as well as the affective and social outcomes of students, survey findings on their self-esteem, and values, and so on), schools can explore the reasons behind the value-added patterns; evaluate learning and teaching effectiveness; formulate and implement follow-up action plans for school self-improvement.
APASO (Assessment Program for Affective and Social Outcomes)
It provides quantitative and objective student performance data depicting students’ performance in the affective and social domains. EDB has chosen ‘attitudes to school’ from the APASO as one of the ‘key performance measures’. The seven subscales in the questionnaires are comprehensive and able to reflect the impact of school life on students’ affective and social development. Schools are to provide the External School Review team with the relevant data for reference.
SHS (Stakeholder Survey)
It is a measure to help schools collect views from teachers, students, parents and specialists (for special schools) on major aspects of school work for school self-evaluation. These include teachers’ view about the professional competence and attitude of the school principal, vice principal and the middle-management; teachers’ view about staff development and staff appraisal, curriculum planning, organization and management; students’ and parents’ view of teachers’ teaching attitude, knowledge and strategies; parents’ view of home-school cooperation; and so on.
TSA (Territory-wide System Assessment)
The TSA is an assessment administered at the territory level by the government. It is mainly conducted in paper-and-pen mode and there is also an oral assessment component for the two languages. The system assessment is only administered at the three levels of Primary Three, Primary Six and Secondary Three. The one for Primary Three was first conducted in mid 2004 and the System Assessment will be expanded to cover Primary Six and Secondary Three in 2005 and 2006 respectively. EDB stresses that TSA is low-stakes in nature. It is to provide feedback to schools about their standards in the three subjects of Chinese Language, English Language and Mathematics, so that schools can draw up plans to increase effectiveness in learning and teaching. EDB claims that the TSA data will help the Government to review policies and to provide focused support to schools. However, most schools are very cautious and concerned about the TSA data. Many of them take the data as Q-mark (Quality Assurance Mark). Some of them use it to publicize their schools.