Abstract
Ideas and concepts about evaluation travel around the globe. Studies of how evaluation models are disseminated, diffused and implemented are important. In this article, we examine an organization with a history of traditions and legitimacy and a successful audit of its own and how it responds to modern concepts of administration where evaluation plays an important role. Based on an analytical framework from organizational theory, we show how an evaluation model has been either adopted, rejected or transformed depending on the local context and consider why central policies have had limited success in its implementation. This article should contribute to a better understanding of the transformation that evaluation undergoes in the journey between and within organizations.
Introduction
The church is of specific interest for evaluation since for almost 2000 years it has relied on internal ways of assessing performance and its internal quality assurance systems have worked reasonably well. Now however the church must respond to the global influence of evaluation. 1 In the article we examine how a) the management of modern principles of administration and b) requirements to implement principles taken from evaluation and quality assurance interact with pre-existing traditions. Non-profit organizations have to strike a balance between spreading efficiently and producing and controlling its internal values (Lindgren, 2001; Braverman et al., 2004; Cairns et al., 2005). Can the church stand on the sidelines or should it also prove its ‘product’ is evaluated and of a high quality? And if the latter, does the church use modern standards for quality measurement and assurance systems? This article contributes to this discussion by examining what happens when modern concepts of evaluation go to church: ‘knocking on heavens door’.
To better understand how ideas are translated, the preconceptions of the receiving organization have to be taken into account. We will demonstrate how three congregations handle a particular concept of evaluation – Kyrkans Q (Church Q), an adapted version of Total Quality Management (TQM). Scott describes TQM as ‘the most significant contemporary attempt to stimulate and institutionalize change’ (Scott, 2001; xvi). In the USA, TQM has been described as a social movement; however, since there is such a diversity of concepts under the TQM ‘umbrella’, it is unclear whether there is an identifiable conceptual core (Hackman and Wageman, 2000). Hackman and Wageman claim that the concept has many meanings. According to its original authors – Deming, Juran and Ishikawa – one of TQM’s assumptions is that an organization should gather and value performance indicators to implement continuous improvements.
The hypothesis is that global concepts of organizing include the notion that ideas travel and transmit rationalized and idealized features of an organization. The empirical question is how are ideas transformed when they reach the local setting? One way of analysing the idea of quality management ‘landing’ is by the new institutional theory that addresses homogenization of organizations (DiMaggio and Powell, 1983). We can imagine how the local practice becomes more like other organizations in the field, but we can also ask questions about strategic responses when studying the institutional influence of travelling ideas (Oliver, 1991). To analyse this influence, we use the theoretical framework of travelling ideas and the ideal organization.
Travelling ideas
Ideas do not possess any kinetic energy: they need some sort of impetus. To follow a travelling idea, we need an analysing model that defines the journey and its organizational settings. This is central in the kind of research (institutional theory) that focuses on the way concepts of change develop and spread. The idea of diffusion as a rationalized process ‘by which an innovation is communicated through certain channels over time among members of a social system’ (Rogers, 2003: 7) is being challenged by theories about implementation, translation and knowledge transfer with great relevance to the field of evaluation (Ottoson and Hawe, 2009). In 1996, Czarniawska and Joerges proposed that attention should be focused on the complex process of translation and how organizations are intertwined. Translation takes place as a process of interpretation where meaning is based on context and experiences. In this process, preferences and policy options are highlighted and interact with existing standards and precedence. This theory of translation suggests that the process ought to be analysed in five phases:
An idea is dis-embedded or decontextualized as it is energized by a vision that it can be useful in another place and time.
The idea is translated as an object, a model, prototype or text.
It becomes a travelling idea. That is, it is exported and used in different organizational settings.
The travelling idea is translated and transformed into action in a local context.
When re-embedded, it is taken for granted as an integrated part of the new practice and becomes more and more invisible in ‘the black box’ of institutionalization (Czarniawska and Joerges, 1996, Erlingsdóttir, 1999).
Johnson and Hagström (2005: 372) believe that these phases are not always ‘heuristically useful’. It is likely that in practice the phases occur simultaneously and continuously and not as these discrete phases. Our interest is the latter part of this journey: the repacking of the idea into a concept and its adaption in a certain kind of organization. There are few studies that address the translation and implementation of evaluation and quality ideas in non-profit organizations (Paton et al., 2000; Cairns et al., 2005; Persenius, 2006).
The ideal organization
Organizations carry ideas, package them, and translate them, often influenced by trends and fashions. An idea when first encountered can be rejected or welcomed, even if it is not in use (Røvik, 2008: 212). The store of tools for reforming organizations and solving their problems is huge, some of which become global super standards (Røvik, 2000). One of these is TQM and in the Church of Sweden TQM has been adapted and is called Church Q. The mere thought that TQM in the form of Church Q might be useful in an old organization such as the church is consistent with the belief that all organizations are alike or that there is a prime way of organizing (Meyer and Rowan, 1991). Organizations (the church included) meet and reproduce socially constructed norms and conventions about how the ideal organization should function. Those norms are institutionalized and powerful and govern the image of an organization. They have an importance that goes beyond their function of mere administrative tools. Instead, popular concepts of organizing gain legitimacy and are spread since they are symbols of how a modern and effective organization should be run, rationally and scientifically.
Sahlin-Andersson and Brunsson (1998) describe this institutionalization of the idea of organization in terms of identity, hierarchy and rationality. Identity is the result of negotiations with members and environments and is closely related to the organization’s domain of action and discretion. To express identity, many organizations work with their logos and brands. Historical heritage can be one source of identity. Fundamental for the organizational identity is the question of boundaries: Who is inside and who is outside? The hierarchy, the structure for authority, shows who is in command and accountable, but also where the boundaries for control are drawn. ‘Throughout the hierarchy, the identity and the organization will be put into action’ (Johansson, 2002: 150, our translation). Rationality follows from the claim that organizations are intentional: they have a mission and objectives to fulfil. Well-informed decisions will structure actions. This includes a strong emphasis on accountability. These three features, fully expressed, sum-up the contemporary ‘image’ of the ideal organization. But when studying different organizations, these characteristics can be more or less developed. They should be viewed as variables. For example, two organizations may be more or less like the ideal organization.
The Church of Sweden
Control is nothing new for the Church. Early routines for the regulation of tithing and catechesis examinations were one of many ways for the priest to control the beliefs and lives of the parishioners. In alliance with the king (the state), the parish was a tool used to discipline the population, an alliance that was successively eroded during the 19th and 20th centuries. Since 2000, the Church of Sweden has been separated institutionally from the state, but retains some societal responsibilities such as burials. From the medieval era, the local parish has had considerable autonomy. The Church of Sweden is heavy at the bottom and light at the top. Concepts new to church praxis include quality control and audit models drawn from New Public Management (NPM). Here emphasis is placed, among other things, on the creation of market mechanisms, the autonomy of departmental units, and contracts that include demands for quality and evaluation (Power, 1997).
In addition, this new approach has encouraged market logics. The Church of Sweden has a clearing system: the church members have the right to baptism, weddings, and funerals in the church of their choice regardless of whether the church is located in their own territorial parish or elsewhere. A parish resident can become a ‘customer’ of another parish. That is, demand rules regardless of parish boundaries. In other ways the church can be described as having adopted a market orientation for residential homes for the elderly, funeral parlours, and schools run by parishes – even though this is still a marginal phenomenon. More and more services of worship are directed towards target groups. These services can be themed or age-specific and can be described as an expression of the market or customer orientation where interventions are distributed through ‘a polyphony’ of various institutions (Villadsen, 2007). This market logic is not an issue particular to the Church of Sweden. When unitary societies become multicultural, religious pluralism affects the church. In such a market, religious institutions rationalize their structures to achieve better use of resources and to attempt to win new members (Berger, 1969).
Methods
A survey sent to the dioceses asked staff members about the parishes’ use of quality evaluation models of various types. The result shows that parishes are restrictive about taking on new ideas concerning evaluation and quality. Only a small number of parishes have experience working with quality evaluation models. The most common is a variant of Total Quality Management (TQM) that has been adapted for the church: Church Q. From the theoretical starting point that ideas are spread across the world and taken up by different organizations, the concept of Church Q has been ‘tracked’ by interviewing ‘concept carriers’ who have packaged them in models appropriate for translation and the local practitioners who have made use of them. For the sake of simplification and overview, the translation will be described as three themes in three phases: from the original idea (theory), via the package of the idea (concept), to the translation into local action
Based upon the survey, we have strategically chosen three parishes and investigated how the idea of quality is understood and interpreted in the context of local actors. The three parishes, which are to some extent similar, 2 had adopted the idea in three very different ways. The social welfare (diaconal) work in the parishes was chosen because it is quite open, visible, and conducted in cooperation with other organizations. We assumed that the quality of its outcome would have to be demonstrated externally and that the quest for evaluations and data on performance would be more immense than in traditional activities within the church.
The case study is based on participative observation of the training of personnel to use Church Q, workbooks, interviews, and documentation from the three parishes. Figure 1 lists the interviewees within the parishes.
The interviewees in the parishes.
Interviews have also been conducted with the former and contemporary Church Q consultant in the Association of Parishes within the Church of Sweden and with a consultant from the Swedish Institute for Quality (SIQ) (Research period Sept. 2005 – March 2009).
Results: Tracing TQM to Church Q
Church Q is a disembedded idea traced to the great TQM family and the Malcolm Baldridge National Quality Award. It was translated by the Swedish Institute for Quality (SIQ) (http://www.siq.se) in the early 1990s and picked up by a consultant at the central church administration. In this original model, accountability plays an important role together with support from managers and employees. A project was initiated to investigate whether the ideas could be useful for the parishes. The SIQ model seemed to be applicable in the church; however, it had to be adjusted to the symbols and language of the church to be understood and implemented. Such adjustment is a usual strategy when a travelling idea meets existing organizational patterns (Czarniawska and Joerges, 1996: 32). For example, when talking about the TQM criterion ‘Leadership’, the Church Q consultant compares it with the ‘The good shepherd’. In the information spread to the parishes, we find several expressions that address motivation and implementation strategies: ‘a support for the management together with the collaborators develop the parish’; ‘a way of thinking that supports development’; ‘guidelines step by step’; ‘through dialogues the strengths in the common work will be discovered’; and ‘improved job satisfaction’ (Nyström, 2005).
As the idea travelled along, it was transformed and named Church Q (Figure 2). Specific material was produced and distributed, courses were offered, and it was tested in a couple of local parishes. Now the church had its own quality-improvement tool just like other large organizations. Today, Church Q is described as a self-evaluation tool, an instrument for improvement, and a device for managers. However, when conducting this research, we soon found out that in spite of eager implementation, only a handful of parishes used it in the intended way. The demand seemed poor. Church Q had been further adjusted and slimmed down at the local level. Some parts were being used as managerial instruments, and sixty parishes (3.3%) used a ‘light’ version as mandatory ingredients when trying to secure funding from the EU. In addition, a specific method-book was designed for the parish board.
The travelling idea.
Parish A has adopted Church Q
The policy of parish A is to create a warm and welcoming atmosphere for its new members. Since the town is growing with many families moving in, the vicar has tried to change the previous conservative culture of the parish. To do this, he uses telephone surveys to elicit information from the church’s members. The vicar describes Church Q as a substantial part of a change programme that is ideologically congruent with his ideas of transparency and democracy. The Human Resource Officer in the parish stresses its focus on reflection, dialogue and goal setting. However, the evaluation part is difficult since it comes with feelings of discomfort when judging the performance of congregational life. One parish priest describes the polemic this way:
When working in the church, you give of yourself. And that makes it so you don’t want to be evaluated.
The parish adapted Church Q as a part of a change programme to evaluate its culture, self-image and identity. The formation of rationality, goals, and evaluation activities is supported by the Church Q process, but the parish has had difficulties using the results and building routine activities.
Parish B has adopted some parts and rejected some parts of Church Q
Desiring to become a natural part of the local community, the parish took responsibility for supporting families and children with psychological disabilities. This parish has adopted other concepts of organizing, such as the Social Audit, to monitor the activities mentioned above. 3 Parish B has also been environmentally certified, and it has developed routines to provide greater transparency. In addition, it has adopted pieces of Church Q. According to the vicar, the Social Audit is a more powerful and complex instrument when monitoring their goals than Church Q since it more effectively focuses on social values and outcomes.
These soft values are not expressed [in Church Q]. It [Church Q] is more descriptive of what you do and how to make it better. (Human Resource Officer)
Church Q was a bit too complex to use although it encouraged discussions and improved mutual understanding among the staff.
Parish C has rejected Church Q
For more than one hundred years, this parish has had a tradition of supporting its poor members. Recently, it has started several new activities, often in cooperation with external welfare actors. The vicar is not convinced that Church Q and modern management ideas can help his parish. His ideal picture of a congregation is ‘ordinary people talking with ordinary people about Jesus’. He is afraid that the congregational life will be too dominated by experts producing services for their clients. His ideal is a congregation where the parishioners support each other.
The vicar describes his way of managing the parish as an ‘on-going conversation’ and that they have a great variety in how to monitor their actions. The parish does not work with steering documents and the board is not interested in evaluations. The administrative chief claims that they are ‘not ripe for Church Q’ and that an adaption would be too expensive.
The local translation
The management in parish A appreciated the Church Q model for its ability to cultivate the organizational culture and the image of the parish. The parish management also spoke about being more customer-oriented, but they did not talk in terms of quality; instead, they spoke of hospitality and creating a culture that welcomed people to their church. They believed that Church Q model did provide them new insights and fruitful experiences. Church Q challenged the staff in another way: The value of order and method, planning and monitoring was realized, but seemed to be far too rational, experienced as something foreign in the culture of a parish. The model was initiated by the parish management, but few notable changes were seen at the diaconal level.
In parish B, the vicar uses many instruments from the modern organization toolbox. After testing Church Q, he rejected most of it. Moreover, in the diaconal work, the Social Audit is a more relevant fit for delivering accounts to the financing bodies. Parts of the TQM idea were already integrated in the parish management. They had tried other concepts and could look at Church Q concept more instrumentally than parish A.
The vicar in parish C specifically defended his view of the identity of the congregation. We believe there was an inherent resistance against the institutionalizing powers, a resistance that is informed by several core beliefs about the role of the church – a congregation does not need marketing; a congregation helps people in the city to be Christians by helping each other; and a congregation is a community, not a staff of religious experts. Church Q, in this view, is mere fashion or a trend inappropriate in a context where the staff members are given a great deal of discretion and decisions are arrived at in informal settings. The evaluation culture is informal, with one exception: in the counselling bureau, financed both by the church and the public administration, the latter demands accounts, statistics and questionnaires of their clients.
The transformation of travelling ideas
While following the travelling idea, we noticed that the concept of Church Q is not used to measure quality. The rational accountability approach is not adapted in the core activities of the three congregations. In the diaconal work, we found no impact of Church Q. In practice, Church Q neither supports evaluation nor is used when marketing the church. Instead, Church Q is used as a human resource management tool. The interviewees claim that some parts of Church Q serve to motivate and strengthen the staff. Its prime function seems to be its positive influence on the working environment. The TQM focus on measurement, performance, and quality is lost in the translation as is the idea that implementation stems from top management. Clearly, the policy of local autonomy is strong. The transformation of the TQM idea is summarized in Figure 3. Parish C is not included since it has rejected the idea.
The transformation of TQM to Church Q.
Analysis
To analyse the significance of the studied local processes, we will make use of the theoretical notions of rationality, hierarchy, and identity presented above. The TQM concepts promote goal rationality while asking for goals, objectives, plans and evaluations. In the translation, the quality idea confirms the hierarchy. The TQM models contribute to define the boarders of management’s authority – the parish priest, the vicar, other supervisors and the staff members who work with Church Q. When Church Q emphasizes the working environment in the parishes, the management focuses on the staff. The organization, in this case the church, emerges as a work hierarchy with the priest as a chief who manages a work organization with professional employees; however, understanding the parish as an organization such as a firm or a corporation challenges the identity of the church.
There is no unified response to this challenge (Figure 4). The kind of TQM concept that is being implemented in the Swedish church for one parish (A) confirms a desire to be modern and for another (C) TQM is seen as trendy or fashionable, not suitable for the old value-led institution. In parish B, Church Q was initially regarded as very instrumental, but was discarded when they found it not useful.
Attitude, action and identity in the parishes.
All the parishes are exposed to the institutionalized ideal of an organization. However, Church Q is only one of many influences. When arriving in an organization, Church Q does not arrive by itself; it seems to be preceded (in most cases) by an emphasis on modern tools of management and organizing. This emphasis is communicated by professionals such as human resource officers. In addition, the authorities act as auditors asking for accounts according to the legislation about working environment and equality.
A factor important to understanding the rejection of external organizational strategies is the structure of the Church of Sweden. While the concept of TQM and similar organizational strategies takes as its starting point the involvement of the highest levels of management and implementation from the top down, this does not function well in the church. The tradition of parochial independence is clearly well anchored. Not even the national bodies of the church are given unquestioned access to local practice. Ideas meant to be implemented from the top down are not taken up other than in the form of local adaptations.
When used, Church Q is described as a tool, or rather a strategy, for the parish priests to take on legal and rational authority with the support of the new models. This complements traditional and institutional charismatic authority with the aim of managing the church according to its traditional institutional demands.
Conclusions
Since organizational ideas travel globally, concepts of evaluation are transformed, adjusted and applied in local contexts although the form and language of the evaluation process varies. Evaluation seems to be a complex phenomenon: it is intertwined in principles of managerialism and it picks up influences and discourses as it travels around the globe. In this landscape, we find organizations that apply and spread ideas about how administration should be conducted. With the desire to be modern, rational, and legitimate, most organizations seem to see evaluation as an important ingredient. Striving for the ideal organization means the application of different norms that support its identity, hierarchy, and rationality. As such a strong norm – a global super-standard – we find the TQM idea.
This journey of the idea of quality – i.e. the translation of TQM to Church Q and into action in a local context – can now be summarized. The ‘original’ business orientation encompassing the assurance of quality, an orientation towards customers and measurement of objectives to increase efficiency, is packaged and transformed as a tool for a better working environment where staff members are encouraged to express objectives. At the same time, the landing of the idea challenges local parishes to identify their ideal organization. Such a tool has to increase the control of resources and members, communicate what is very special about the organization, express distinct objectives, and show that the local management can be responsible and accountable.
In a church setting, this process can be ambiguous. Because the identity of a church is strongly supplied by the tradition and the historical roots in every parish, the discretion and freedom of adapting or rejecting these travelling ideas (even if they are ‘global super-standards’) are, in the short run, significant. The case studies show that the parishes’ diaconal practice and probably even other core activities can be screened from external insights and evaluation attempts. The parish priests act as gatekeepers, but buffers in the form of secretarial routines can also deflect external evaluation attempts by disconnecting and deflecting.
The last stage in the theory of travelling ideas – that the idea will be re-embedded and institutionalized – does not seem to be applicable in our case. The travelling seems to have stopped. Despite labelling and adjustments, the quality model challenges the view of what a congregation represents. However, pieces of the idea still appear to be on the move albeit toned down and spread out. Church Q is accepted as a tool to improve the working environment. The process – when evaluation processes are transformed from TQM to Church Q to an adapted form of Church Q – might best be described as transformation characterized by the local interpretation of organizational identity.
This research has been restricted to three parishes in Sweden. It is not possible to generalize the results to other cases. There is a discussion to be had regarding whether it is at all possible to generalize research results from studies of translation (Scheuer, 2006: 3); are all change processes unique, or do variables in our institutionalized world exist that may determine or influence the content and direction of translation processes? However, we believe it is fruitful to make analytical generalizations and regard the institutional analysis as a promising way to examine such issues. Further research should investigate the negotiations between organizations and their environments, but also the negotiations inside each organization. Evaluation does not always trigger pressures for better performance, but is undeniably used for the management of power. The analytical concept for institutionalization (rationality-hierarchy-identity) can be developed to better understand the institutional pressure on voluntary/non-profit organizations, as in our case, with strong ideological identity. In churches, the ideology is often represented by the ministry but is also heavily dependent on their historical roots.
The institutional environment, the internal structure, and the local identity are all examples of what a professional evaluator has to take into account, each being one piece in the mosaic of evaluation and a subject for the evaluator’s contemplation.