Work-integrated learning and health literacy as catalysts for Roma empowerment and social inclusion: A participatory action research

Abstract

Roma people all over the world have often been subject to prejudice, stigma, discrimination and oppression. Many Roma have little or no education, which in combination with other factors often leads to unemployment and marginalization. Based on a case study in Sweden, this paper proposes an approach that can be used in participatory projects aiming at improving the living situation of marginalized people. In such an approach, the Roma themselves would be the leaders in all activities concerning the group. The first aim of this paper is to describe how health literacy and work-integrated learning functioned as empowering tools for a Roma-led inclusion process within a participatory action research framework. The second aim is to discuss and reflect on the use of participatory action research in community work with Roma based on the experiences of the participatory action research participants. The findings indicate that work-integrated learning may be a worthwhile approach for increasing the individual empowerment and self-led social inclusion of vulnerable people. However, the obstacles of structural discriminatory nature hindered the project to reach its full potential in its intended emancipatory goals.

Keywords

Empowerment health literacy participatory action research social inclusion Roma people work-integrated learning

Introduction

The Roma are the largest ethnic minority in the EU (United Nations Development Programme (UNDP), 2013). Although most Roma are EU citizens, many have been trapped for generations in a vicious cycle of social exclusion and discrimination. Based on a study in South Eastern Europe, Milcher (2006) suggested that such exclusion was associated with discrimination based on Roma ethnicity and was an independent risk factor for Roma poverty. Roma in the EU suffer from a lack of education. Milcher (2006) showed that only 66% of poor Roma were literate, in comparison with 92% of the poor majority population. According to a survey in 11 EU member states (European Union Agency for Fundamental Rights; FRA & UNDP, 2012), 90% of Roma live below the poverty line, only 15% of young Roma adults have completed their upper-secondary education, one out of three is unemployed and many lack knowledge about their rights as stated in the EU laws. Research shows that European Roma have worse health and a life expectancy that is 10–15 years shorter than non-Roma populations (Parry et al., 2007; WHO, 2012).

The Roma are a heterogeneous group, spread all over the world, with e.g. various religions, traditions, cultures, ways of life and countries of origin (Hancock, 2013). The common nominator for all Roma is the joint language, Romani Chip. The language, however, has different dialects depending on the country of origin (SOU, 2010). As the Roma identities and communities vary, the concept of ‘traditional’ Roma is false and stereotypical and a creation of intergovernmental bodies and governments that has been adopted by civil society (Regeringskansliet, 2014). Today there are more mixed Roma that could be considered as traditional than Roma living in entirely isolated communities (Nicolae, 2012). Professor Ian Hancock (a well-known Roma himself) reminds that the first thing when interacting with Roma is to remember that the Roma are people just like anybody else (Hancock, 2013, p. 91). However, ‘there are great differences among Roma groups, and what might be “more Romani” in one rather than another group may turn out simply to be custom adopted from the local non-Romani population’ (Hancock, 2013, p. 107).

Approximately 50,000 Roma live in Sweden (Palmroth & Hermansson, 2006; SOU, 2010). The figure is only an estimate due to prohibited ethnic monitoring in Sweden and also because of ethnic self-ascription. Because the Swedish welfare system (e.g. through social subsidies system) ensures that no citizens fall under the poverty line (SOU, 2010), the situation appears to be somewhat different for Roma in Sweden compared with many Roma in other European countries. The Roma in Sweden were awarded a status of a national minority in 2000, which means that the civil and human rights of this group are protected by law (SOU, 2010). Even though human and minority rights and antidiscrimination laws are in place, many Roma live under poorer conditions and participate less in the democratic processes than the non-Roma population due to low level of education and lack of awareness of their rights (Eklund Karlsson, Crondahl, Sunnemark, & Andersson, 2013; SOU, 2010). Throughout the 1900s, the Roma were subjected to several comprehensive surveys by the Swedish state and municipalities (Regeringskansliet, 2014). Unfortunately, many of these nationwide surveys of the Roma together with the research on Roma by the scientific communities built up and confirmed, rather than challenged the negative and persistent stereotypes of Roma and hereby strengthened the prevailing prejudices. These views were a starting point for many of the Swedish governmental laws, institutional regulations and actions (Regeringskansliet, 2014). Policy measures of the past have thus not sought to change the unequal relationship that has existed between the Roma and the non-Roma population in Sweden. Antiziganism is still a problem in Sweden (Regeringskansliet, 2014), and it contributes to Roma low trust in the non-Roma society and their tendency to keep outsiders in a distance (Hancock, 2013). An important prerequisite for overcoming the lack of trust and antiziganism is not only actions challenging the stigmatized thinking about the Roma but also ‘acknowledging and taking responsibility for the abuses and violations committed by the Swedish state against Roma during the past years’ (Regeringskansliet, 2014).

The consequences of the marginalized situation of many Roma in Sweden are a general sense of powerlessness and hopelessness (Eklund Karlsson & Crondahl, 2010). Roma people themselves identified education and employment as key determinants to health, well-being and quality of life and as prerequisites for participation on equal terms in society (Crondahl & Eklund, 2012; Eklund Karlsson et al., 2013). The Roma people in Sweden, however, have been regarded as a helpless group in need of expert help and action from authorities. They have usually not been given the opportunity to actively participate in adjusting efforts to their needs and situation (Palosuo, 2008; Regeringskansliet, 2014; SOU, 2010). The discussions and ambitions have not been derived from the Roma people’s own norms and traditions, but from norms and traditions that are perceived as unilateral demands of the non-Roma society.

An alternative way to approach the Roma situation is to enable Roma to take the leading role in initiating processes and activities concerning the group (Eklund Karlsson & Crondahl, 2012; Palmroth & Hermansson, 2006; SOU, 2010). These ideas were the basis for a participatory action research (PAR) project initiated by a researcher at the University West (in 2009) who had an earlier established contact with one of the key persons of the local Roma community. Trough parallel training and working (= work-integrated learning (WIL)), the aims of the project were to empower the Roma to dispel the prevailing stigma and stereotyped thinking about Roma among the civil servants, increase Roma participation in the society and facilitate a Roma-led social integration. The project was conducted as a PAR, aiming both on societal change and capacity building of Roma.

The first aim of the present paper is to describe how health literacy and WIL functioned as empowering tools in the Roma-led inclusion process within a PAR framework. The second aim is to discuss and reflect on the use of PAR in community work with Roma based on the experiences of the researchers (i.e. the Project Leader, the Main Coordinator and the Senior Researchers) and the co-researchers (the local Roma coordinators).

Description of the PAR project

In this study, the preliminary theoretical assumption was that a PAR and WIL approach may facilitate empowerment. The concepts of empowerment, health literacy, WIL and PAR formed the study’s theoretical underpinnings.

Theoretical framework

Empowerment may be a worthwhile strategy for combating poverty and improving population health (Wallerstein, 2006). Participatory processes are at the heart of empowerment. However, participation is insufficient without building the capacity for community organizing and developing strategies for individual decision-making and advocacy. The use of popular education (Freirean education) has been recognized as an effective tool for this capacity building (Wiggins, 2011). Freire (1996) emphasized that people first need to be ‘critically conscious’ about social inequality, injustice and oppression in order to change this situation and transform their learned helplessness or oppression into mastery of their life. The Freirean education builds on the participants’ knowledge and personal experiences, connects these to larger social realities and supports the participants in working collectively to change their reality (Freire, 1996). The critical consciousness comprises the ability to reflect on the conditions that determine people’s lives, the perceived strengthened capacity to influence in political and societal change and act upon it (Watts, Diemer, & Voight, 2011). Wallerstein et al. (2008) suggest that empowerment should not only be investigated as individual or group change but also as change in the social conditions and setting where it takes place. The WHO incorporated the term empowerment in its definition of health promotion: ‘… the process of enabling people to increase control over, and to improve, their health’ (WHO, 1986). The word ‘enabling’ implies that the only way for a person to be empowered is through acquiring more power; one cannot ‘be empowered’ by others. Empowerment has been described both as a process (the creation of a professional relationship in which the individual, group or society takes control over the change process and decides both its targets and tools) and as a result (achievement of control over the factors that affect one’s quality of life) (Rappaport, 1985; Tengland, 2008).

Health literacy has been claimed to have a crucial impact on empowerment (Mårtensson & Hensing, 2012) even though this impact has not been verified with empirical evidence (Crondahl & Eklund Karlsson, 2016). According to Nutbeam (2008) health literacy is developed through continuous learning in different contexts and learning processes and results in increased self-efficacy, changed motivation and attitudes towards health behaviour, and increased knowledge and understanding of the determinants of health. The level of health literacy has a direct effect on a person’s ability to absorb and act on health information and allows individuals to take greater control over their own health, their family’s health and the group’s or community’s health. In the health promotion context, health literacy is regarded as more than being able to read written material which Nutbeam (2000) calls functional health literacy. More advanced levels of health literacy are the interactive health literacy (the cognitive and literacy skills that are needed for social skills to improve person’s capacity for interactive communication) and the critical health literacy comprising person’s ability to critically analyse and apply health information for achieving better control over individual’s life situation (Nutbeam, 2000, 2008). Abel and Bruhin (2003) expand the traditional individualistic definitions of health literacy to include social engagement in broader health issues. They emphasize the importance of lay knowledge related to health, welfare and everyday experiences. Abel and Bruhin argue that health literacy also includes the values, norms, attitudes and orientations that affect people’s perceptions of health and disease and that health literacy increases the chances of changing health-related living conditions. Crondahl and Eklund Karlsson (2015, 2016) suggest that for health literacy to be critical to empowerment it must focus on the social determinants of health with an emphasis on the individuals’ and communities’ own perceptions of what health and quality of life is for them. Crondahl and Eklund Karlsson (2016) regard health literacy as a catalyst for empowerment.

WIL means the combination of formal and informal education (Murphy & Calway, 2008). Research has shown that theoretical knowledge acquired from formal education can be improved through informal learning in everyday work and vice versa. The WIL approach enables and enhances the participants’ understanding and performance; allows for deeper learning (Freudenberg, Brimble, & Cameron, 2010; Murphy & Calway, 2008) and increases self-confidence, hope for reaching personal goals, self-efficacy, self-esteem and self-image (Bates, Thompson, & Bates, 2013; Purdie, McAdie, King, & Ward, 2011). Furthermore, the WIL approach develops the participants’ professional, discipline-specific, problem solving and analytical skills, as well as their confidence and maturity (Bates et al., 2013; Freudenberg et al., 2010). One of the main goals of WIL is to help the participants gain real-life work experience. Such experience prepares the participants to face challenges at the workplace and makes them more employable (Purdie et al., 2011). WIL enables more autonomous study and can be considered as an active learning process involving both the performance and the reflection of real tasks. The work experience provides the participants with both tacit and articulated knowledge, which in many cases only can be accomplished by being on-site at a workplace (Choy & Delahaye, 2011).

PAR incorporates the ideas of participation, collaboration and transformation impacting into cyclical loops of action and reflection through learning and practice (Minkler & Wallerstein, 2008; Reason & Bradbury, 2001; Wallerstein & Duran, 2006). This idea is grounded on the assumption that everyone has capacity to construct knowledge through communication and interaction with others, regardless of culture, ethnic or academic background (Puigvert, Christou, & Holford, 2012). This requires a focus on power issues regarding cooperation and learning about how to allow all voices to be heard, how to encourage critical collaborative dialogue between the participants (Ripamonti, Galuppo, Gorli, Scaratti, & Cunliffe, 2015) and how to create shared ownership (Springett, Wright, & Roche, 2011). The authors of this paper see these as elements of communicative and critical health literacy.

The study design

The project was conducted within a PAR framework. Knowledge needed for empowerment was to be generated in the interaction and critical dialogue between the people involved, both Roma and non-Roma. An important element was that the Roma participants would get their share of the resources of the project from the very beginning: The activities provided for the local Roma community and the local coordinators salaries were financed by the project. The Roma coordinators were employed in the project on equal terms as the rest of the project group and were not just objects of actions. This is important in enabling the marginalized communities to participate fully in projects and ensure that the projects are addressing issues that are high on the participants’ own concerns (Wallerstein et al., 2008). The initial plan was to employ a critical mass of Roma who, through WIL, would be educated according to the principle of ‘training the trainers’. The coordinators were first to be empowered in order to be able to empower the others in their local Roma community for social action.

Context

The project took place in the four municipalities of West Sweden, where most of the Roma live in this region. These municipalities agreed to be partners in the project and appointed one civil servant in each of the municipalities as a focal point for the project. Additionally, a Folk High School in West Sweden became one of the partners after several brainstorming meetings and negotiations between the researchers and the staff. This Folk High School is an adult educational institution that is managed and staffed mostly by Roma, and the majority of its students are Roma. The University West was the host organization of the project and the employer of the project group.

Participants of the project

Approximately 360 Roma were involved in the project from 2009 to 2012 (through various activities organized by the project). The figure includes also the employed local Roma coordinators. A total of 1299 public sector employees from the same four municipalities (mentioned above) took part in the lectures on Roma issues, which was one of the WIL practices performed by the coordinators.

A two-step research process

The project was made up of two phases (Figure 1). The initial phase (Phase I) included planning, mapping, pre-course and analysis in which the Roma communities were involved on voluntary basis. The results of the initial phase were the basis of the second phase (Phase II), comprising the implementation of the planned competence development activities for civil servants and community organizing activities with the Roma communities.

Figure 1.

The phases of the project.

Phase I: Planning, mapping and pre-course (September 2009–July 2010)

As the initiators of the project (i.e. the project leader and the senior researchers) were not themselves members of the Roma community/group, a key step was to involve a Roma liaison person before starting the project to serve as a ‘gatekeeper’ and mediator. The Folk High School provided one of its Roma staff members as the mediator who introduced the project group to the Roma communities, made contacts with the Roma people and engaged them in the project planning and activities. The mediator was also a member of the project management group.

During the Phase I, the initial project group was established and consisted of a senior level researcher/project leader (responsible for the scientific guidance of the study, curriculum development and WIL teaching), a main coordinator/action researcher (responsible for the WIL activities, the managing of change and research operations) and three senior teachers/researchers in the field of public health and social sciences (involved in the teaching and research activities).

Mapping of Roma perceptions on health and quality of life was conducted by the senior researcher group. Approximately 200 Roma from the above-mentioned four municipalities in West Sweden participated through interviews (n = 30), surveys (n = 100) and discussions and dialogues in various workshops (n = 70) about the Roma life situation and issues concerning the Roma. Stakeholders and civil servants (about 30) from the public sector and educational institutions participated through discussions and dialogues. The data were analysed in collaboration with the Roma and the project researcher group (Eklund & Crondahl, 2010; Eklund Karlsson et al., 2013; Hassler & Eklund, 2012). The analysis revealed that education and employment were considered to be the keys to a better life and the determinants of health and well-being. Health was perceived as a resource and an ability to self-manage and was seen through a family perspective, rather than through individual experiences, personal needs and wishes. Obstacles for Roma empowerment were identified as lack of education, unemployment, low self-esteem, isolation, prejudices and discrimination, and constrained thinking in the Swedish welfare system. These findings were the foundations for phase II.

Partly in parallel with the mapping, a pre-course on societal organization and empowerment was organized for interested Roma, comprising 20 afternoons over 10 weeks (Eklund & Crondahl, 2010). The course content was built on the Roma health perspective as revealed by the mapping described above, including issues perceived by the Roma to be connected with health and quality of life, such as power, equality, gender equality and discrimination. The course also included research exercises like data collection (i.e. how to interview and make questionnaires), analysing and communicating the results. These results were added to the foundations for phase II. Those who passed the pre-course examination successfully were encouraged to apply the positions of local coordinator for Phase II, implementation (see Figure 1). However, the positions were also open to other Roma.

Phase II: Implementation (August 2010–August 2012)

Employment as local coordinators and description of their work

Seven coordinators were employed, five of whom had participated in the pre-course. The selected coordinators represented the Finnish and Balkan Roma Groups who, in addition to Swedish, spoke Romani language and the language of their country of origin.

At the time of employment, most of the coordinators were novices in community organizing and needed theoretical and practical knowledge. Their WIL practice (Table 1) consisted of two main elements that ran in parallel:

Planning and implementing competency development lectures (totally about 40) about Roma history, traditions, culture, health and life situation that were directed towards employees in the public sector and tailored to the needs of the audience.

Practicing how to engage the local Roma community in social action, make attitudinal and behavioural changes, and influence changing social norms and practices such as discriminatory actions and structures (e.g. joint town hall planning meetings with Roma and civil servants and policy makers; contributing and speaking in conferences and workshops; participating in debates; contributing in radio programmes and communicating Roma issues via local newspapers; organizing various community group activities such as bowling, basket, gym for ladies as well as joint theatre visits, movie nights, etc.).

Table 1.

Main WIL activities

(T = Theory; P = Practice)
T 1: Community organizing – focus on vulnerable groups
P: Planning and information meetings with local communities (total 36)
T 2: Improvisation theatre for pedagogical development
P: Lecturing on Roma issues (total 17)
T 3: Project planning and community Health Promotion (HP)
P: Community organizing in practice I: Future workshops with local communities
T 4: Empowerment and participation to combat social exclusion
P: Participating in anti-discrimination workshop
T 5: Pedagogics in HP – making oral presentations – forum theatre
P: Communicating through media (writing newspaper articles)
T 6 and 7: Community analysis on discrimination and the multicultural society. Part I and II
P: Bilateral knowledge exchange workshops with various institutions in Finland, Kosovo, Greece, Hungary, Scotland
T 8: Pedagogical methods for team working
P: Community organizing in practice II: weekly HP activities in all four communities, bowling, gym, movie evenings, theatre, etc.
T & P: Participating and speaking in national conferences
T 9: Working with people living in exposed situations
P: Mid-term evaluation workshop and debate
T 10: Pedagogical approaches to skills development
P: Lecturing on Roma issues – elaborated presentation (total 26)
T 11: Power, gender, ethnicity and diversity
P: Excursion to Roma Culture and Information Centre, Malmö (Sweden)
T 12: Inequality and health
P: Contributing in national conferences (total 3)
T 13 & P: Evaluation with external evaluators
T 14: Media and society
P: Communicating through media (contributing in radio programme)
T 15: Project: Organizing a seminar
T & P 16: Performing the seminar: Inspiration day for tackling exclusion
T 17: Social identity, participation and learning
P: Contributing international conference, Migrant and Ethnic Health
T 18: Community-based HP and activism
T 19 & P: Internal evaluation and focus group discussions
T 20: Project summary and reconciliation

The WIL training (see Table 1) of Roma coordinators was directly focused on supporting them in their empowerment process including the delivery of these work tasks successfully.

Conducting the project within a PAR framework was a mean to making the voice of the local Roma coordinators loud throughout the whole project. The Roma coordinators were also included in the project management group in the implementation phase. All the activities, decisions and research activities of the project were thoroughly discussed with the local coordinators who also conveyed the views of the wider Roma community (through regular meetings with the community representatives) in the ongoing planning. The project activities were then adjusted to fit both the learning needs and the norms and values of both Roma and non-Roma. For example, more days were organized to be devoted to the theory sessions, more time for joint discussions and share of experiences, more planned time to read literature about Roma, specific skills training about how to engage people in social action were organized, the WIL activities were focused particularly on children, youth and women as suggested by the Roma coordinators. To support the work of local coordinators the participating municipalities organized offices for the local coordinators for their work (as requested by the coordinators).

Data and analysis of the implementation phase

The data were collected during 2010–2012 including self-evaluation reports and written essays of the local coordinators about their WIL experiences, communicative observations, project groups’ workshop notes on the critical dialogue sessions, 21 monthly project reports, the project plan, the syllabus of the WIL training programme and notes from lecture and dialogue – occasions with the involved civil servants from the participating communities. We also included the intuitive, experiential knowing (of the project group) that existed beyond the ‘rationalistic’ data (Bradbury & Reason, 2003). The data were considered as a whole and analysed using a hermeneutical understanding in a triangulating fashion.

The analysis was a reflexive process that circled between the empirical data and theory (Ripamonti et al., 2015): The evolving theoretical ideas and pragmatic explanations were critically discussed by the group of involved researchers (including the co-researchers/local coordinators) and further elaborated or rejected (Puigvert et al., 2012) before ending up in the working model suggested in this paper. The analysis was inspired by heuristic research (Kenny, 2012; Moustakas, 1990) which explores phenomena in depth from the researchers’ experiences of the investigated phenomena (Hodkinson, Biesta, Gleeson, James, & Postlethwaite, 2005). The ‘self’ of the researchers was present during the whole interpretation process, in which tacit knowing and intuition are essential. Of equal importance was that the findings would help to improve Roma health, access to education and their self-led integration efforts (i.e. catalysing the social change that is the main goal of participatory research).

Ethical clearance

The employed Roma coordinators were provided written and oral information about the project and the research processes involved, after which they gave written informed consent prior to their employment. An ethical clearance for the project was maintained by the local ethical committee of the University West (2012/903 B22).

Findings

The Roma coordinators’ experiences on the WIL and their perceived empowerment

The findings indicate that the functional and interactive health literacy of the local coordinators improved during the project. An important factor behind this improvement was, according to the coordinators themselves, the WIL approach of the project. As they were first trained in relevant topics and then had the opportunity to directly test this out in reality, they felt more competent in performing their work.

During the project we have had theory sessions on different topics. This created prerequisites to illuminate and discuss the experienced problems of our coordinator group and the other participants (from outside our project group). I think that everybody contributed and shared their experiences. (Coordinator 5, 2012)

For example, after each theory module, we applied it in our lectures for the public sector. … This (the lectures) served as combined training and work. We had the opportunity to ask and receive positive feedback from our colleagues and to improve our next lectures and the work we performed. (Coordinator 2, 2013)

The coordinators talked about the positive effects of working in groups. As they never had to meet with the public sector employees or with the local community all by themselves, they felt more confident in performing their work. Thus group work was perceived as inspiring, developing, strengthening element which increased their self-confidence.

When you try it out in practice, then you get to see if it is right, suitable. … Each of the project education sessions were designed in accordance to our needs. And, after each session, we tried it out in practice and then we got back and we discussed and reflected whether or not our experience matched the theory. (Coordinator 2, 2013)

An essential issue pointed out by the local coordinators was that they did not only participate as local coordinators, but they were employed as such. This is an issue of egalitarian and ethical implication; being employed made the participants feel equal to the non-Roma project employees.

… in this Project we have been colleagues and we have been at the same level even if X was a manager and I an employee. I was treated as a co-worker, not as a worker. … I think I have been treated with respect, and I have tried also to show respect back. (Coordinator 2, 2013)

Being employed and obtaining a salary was also important as most of the participants had a family to provide for. Moreover, being employed was a new experience for some of the participants, as this was their first paid job. Thus, being paid strengthened the participants’ experience of authorization of being involved through their perceived strengthened competence.

The WIL approach was not the only factor contributing to learning and personal development; the combination of WIL with PAR approach was just as important (i.e. that the WIL participants influenced the content of the project and that they had an active role also in the research activities, such as data collection through photo voice, polls, interviews, etc. from the Roma community, analysis of these data and communicating the results in their lectures for the civil servants).

We get to learn things that give us the knowledge and skills to perform and accomplish different tasks and work. We are participating in decision making. All these things together increase our self-esteem to speak in front of the people sitting in high positions about Roma issues; about the discrimination and such. We give concrete proposals on what might be done to reduce the problems. These things is it we who do, we the Roma! … Roma people taking the power of their own lives! (Coordinator 1, 2012)

To be trusted by the non-Roma colleagues and to perceive their trust perceived to be both unusual and important.

It is not easy to take the power of your own life when you have been discriminated and trampled on so many times in your life. But when a person from the non-Roma population shows you authentic respect and supports, then you may trust that person, and then it is easier to go on and to not give up. (Coordinator 7, 2012)

The essential participatory perspective of the project was upheld and confirmed by the coordinators. They expressed the experience of having been involved in the project processes from the very beginning, including being part of the planning and choice of the themes for the theoretical lectures.

We contributed in the planning of the project from the start. We were involved in deciding which training modules we should have during the two-year period and how to prioritize them. So we ensured that they (the training modules) were suitable for the Roma situation. Thus, the training modules were tailored to our needs. (Coordinator 2, 2013)

The active role that we Roma have had in this project makes us feel part of the efforts that are being done for the Roma. There has always been so called “experts” among the non-Roma society who have “known” what is the best for the Roma. But this project is different as we mobilize our own local community, we lecture about our culture, traditions, manners and language. It is we who are the experts. This feels important for us Roma who for so many years have not been allowed to participate in the issues concerning ourselves. (Coordinator 5, 2012)

The involvement in the decision-making made the coordinators feel like they were trusted and independent, which strengthened them:

We were involved in the decisions and we were consulting each other. We were allowed to make our own decisions in our local community. That makes me feel participative. It has strengthened me that the project facilitators trusted me to make my own decisions and that they listened to me. (Coordinator 4, 2013)

The working climate was perceived by the local coordinators as good, open and permissive, making them feel more like friends than colleagues. They felt they were treated like co-workers and equals.

I am very happy that there are people who consider us Roma people too to be individuals like all others. (Coordinator 5, 2011)

Whether or not you dare express your opinion depends on the people around you. When there is a friendly and open climate, then you also dare to speak up about things. In this project, I have not been afraid of doing that. (Coordinator 1, 2012)

I have had in my mind that the project leader was the boss for one part of the project, but I was the boss of another part of the project. So I have felt that we are two parts who have met and that we complement each other. (Coordinator 2, 2013)

Key issues and strengths in the project were the emphasis on a bottom-up perspective and equality. With a starting point of different cultures, norms and values, the process was not free of challenges, thus highlighting the importance of such approaches in reaching an understanding of each other’s views, instead of making them obstacles.

The outcomes of the project from the Roma coordinators’ perspective indicate that the Roma coordinators perceived the WIL approach as fundamental for their personal development and for their ability to contribute to the common good as well as their self-led social inclusion.

A working model of empowerment for social inclusion

The empowerment model (Figure 2) this paper suggests below is a result of the critical dialogue process and analysis of the project by all the involved researchers and co-researchers. Conceptualization of health literacy as an ‘asset’ (Nutbeam, 2008) was the initial starting point for the elaboration of the model. According to Nutbeam conceptualization has neither been tested through systematic research nor turned into practice. Thus, our elaboration of the model is an attempt to fill this gap. The figure below illustrates and summarizes our understanding of the interaction of the various conceptual elements of the approach and how they intertwine in a way that may lead to increased health literacy and empowerment of participants. The suggested working model is both process oriented and outcome oriented, indicating both vertical and horizontal operations of action. The main goal of the project was the Roma coordinators’ increased empowerment in which health literacy was regarded as a determinant of empowerment (Crondahl & Eklund Karlsson, 2016).

Figure 2.

Illustration of WIL and health literacy as catalysts for empowerment.

The development of empowerment and health literacy occurred in three parallel tracks (Figure 2): Track 1: In close collaboration with the Folk High School, those local coordinators who had not graduated from primary/secondary school and/or high school before attending the project were able to finish their school during the project; Track 2 (a, b and c in Figure 2) consisted of the tailor-made vocabulary education including 20 two-day training sessions (WIL theory) provided by the initial project researchers group (about community organizing, teamwork, knowledge of health determinants and skills and knowledge in advocacy, communication, action for social change, health promotion as well as research skills and adult pedagogics); and the Track 3 (d, e and f in Figure 2) consisted of WIL practice that was conducted in the Roma coordinators’ own communities (see details in Table 1).

The framework comprises the idea of facilitating the marginalized Roma’s fast track access to work and further education (which are the key determinants of health perceived by the Roma themselves) without waiting (often several years) for the formal graduation diploma from school: The project provides the work and education platforms and at the same time through collaboration/agreement with the formal education system (in this case the Folk High School) makes it possible to complete school.

The analysis of the data revealed that the coordinators’ interactive health literacy skills were increased as a result of the capacity-building approach (Freire, 1996; Wiggins, 2011) that was applied to the mentioned training sessions (a, b, c in Track 2). However, the progression between the health literacy levels was not only dependent upon cognitive development through the theoretical elements (a, b, c), but also upon exposure to different forms of communication with the local community and target audience (civil servants) and ‘testing’ and messaging their learned knowledge (including their own research activities and results) in their practical work (d, e, f).

Examples of the work with ‘engaging the people in social action’ expressed by the coordinators themselves (d, Track 3):

“Awareness rising” means that we increase the Romani people’s awareness of how the situation looks for Roma today. We involve them by asking them to tell us what problems they experience in society and involve them later in the joint work changing this. We organize ‘Futures Workshops’ in which they are active, and write down the various problems they are experiencing, and suggest solutions. Then we work together to tackle these main problems. We do not do stuff for the Roma, but we give them the tools so that they themselves can handle situations and problems. This is how we empower them. (Coordinator 3, 2013)

Examples of the work connected to ‘changing behaviour and practices’ (e, Track 3):

I want to be a role model for the Roma women… so that they could be able to come out into the society and strengthen our group. I will try to support other Roma people to be able to carry out and organize their own activities around health, traditions and culture. My job is to support our youth to complete their education, as education means a better future for us Roma. Education means power. (Coordinator 3, 2013)

In the Municipality, we have chosen to work with Roma women because they are double discriminated against. We do different things (activities) to make them to leave their home and take part in the (non-Roma) society and in this way to improve their health, both mentally and physically … and step by step, enable these women to pursue their own issues in society. (Coordinator 6, 2013)

Examples of the work on enabling ‘participation in changing social norms and practices’ and strengthening the Roma self-confidence expressed by the coordinators (f, Track 3):

…we have convened briefings and a range of meetings in various contexts in order to inform different government agencies and officials about the Roma issues that are important to us. We have been lecturing about us Roma to try to reduce the prejudices that prevail about us. On these occasions, we have also tried to talk about the discrimination that often occur in different contexts. This is the attempt to reduce prejudices and preconceptions. In our lectures, we often take up our own experiences of being a Roma and we try to explain e.g., that we should not be regarded as a homogeneous group. We Roma have for many years lived under discrimination and persecution, and this affects our confidence in the authorities and officials. What was good with the Project was that it strengthened our self-confidence and self-esteem. (Coordinator 2, 2013)

The sharing of experiences strengthened the participants’ capacity for collective action and social change (critical health literacy). Together with the course participants, situations where the participants had experienced injustice, oppression, a poor reception or discrimination by desk officers, authorities, etc. were discussed, analysed and dealt with using different techniques, such as role-play, forum theatre (Hamel, 2015) and other active learning approaches.

I was a little hesitant when the first time I’d go there (bowling hall). So that they would not say when we get there, and when they see who we are, that “no, sorry you cannot come here”. … We are so accustomed to become disappointed because of we are denied to enter places. And it's exhausting. But through this project, I have learned a lot. I’ve been strengthened enormously. And all this that you’re not so afraid anymore: When coming to a new place - do we really need to tell that we are Roma, then? No, we don’t! Now I think instead that you should be proud of what you are. (Coordinator 4, 2013)

The findings indicate that the project itself was a platform that facilitated higher level learning processes (interactive and critical health literacy) through the opportunity to use the acquired knowledge in their practical daily work (lecturing about Roma for civil servants participating community meetings conveying the Roma community views in those and organizing activities for the Roma community to assist in bonding connection between the Roma and non-Roma people). The results indicate that increased critical health literacy developed through their improved abilities to express themselves and conduct a dialogue with their earlier oppressors and to mobilize the local Roma community for social change (WIL practice, track 3). This allowed the coordinators to experience a greater control over their own lives and integration processes.

Discussion

The discussion and reflections on the use of PAR in the empowerment of Roma communities was guided by Wallerstein et al.’s (2008) framework on how participatory processes influence or predict intervention outcomes. It includes considerations about the contextual factors that shape the nature of the research and the partnerships, the structural and relational group dynamics within the project, the intervention and its design, and outcomes.

About the contextual and governance challenges of the PAR

The contextual and governance factors influenced the participatory nature of the project and the degree of emancipation and the participants’ freedom to integrate their learning into their work. The following experiences of the project illuminate how the Swedish democratic context and processes in a society, initially meant to ensure equity, became obstacles of structural discriminatory nature instead (see e.g. SOU, 2010): The initial plan was to employ those Roma who were farthest from the labour market. However, as the host organization’s (the University) employment regulations and rules needed to be followed, the best-qualified applicants had to be employed instead. Furthermore, as a result of the funding body’s regulations (e.g. who were regarded as ‘participants’ of the project), the Roma were stuck in the middle of the financing body’s criteria. Thus, the Roma coordinators and the Roma community were not eligible for funding for all the project’s activities. The financing body of the project demanded a clear distinction what is meant with the ‘participant’ of the project: Only the Roma coordinators and the public sector employees in the partner municipalities were eligible as recipients of the competence development lectures delivered by these local Roma coordinators. In other words, only these two stakeholders were counted as ‘participants of the project’. The wider Roma community (i.e. the Roma living in the involved three municipalities in West Sweden) were, according to the rules of the financing body, not eligible as the ‘participants’ of the project. This was because the call of the grant from the financing body was targeted to competence development of civil servants. The Roma people did not belong to this category as almost 100% of them were unemployed and many without formal education. Consequently, the application of the acquired empowerment skills of the Roma coordinators towards the Roma community was to be left as the coordinators’ (and the researchers’) free-time activity. Sticking into these rules was the funding requirement of the financing body.

To limit the effects on the Roma as a result of the rules and regulations of the financing body, the project had to navigate the course of the project in a way that would increase the financial equality and equity of the Roma within the project. Examples of this were efforts to avoid lowering the salaries of the Roma coordinators (due to lacking formal education) and in influencing the financing body to allow us to increase the number of Roma to be employed. However, the restrictions and ‘participant’ exclusion criteria by the financing body also reflected the original participatory idea of the project. As a matter of fact, the top-down steering by these organizations was one of the factors that hindered the project’s full potential in reaching its intended emancipatory goals. Based on the document data, it became obvious that the project, at the end, was spending more time on fulfilling the regulations and bureaucratic requirements of the financing body and steering committee of the host organization than on the original purposes of the project (which was facilitating a Roma-led social inclusion and empowerment). The prevailing ‘Moment-22’ (‘Catch-22’) was that the financing body called for innovations (in our case the shifted roles of the powerless and the powerful), but in order to ensure their financial support, the innovation had to be transformed to be fitting to the existing thinking and concepts which did not allow innovations.

About structural and relational dynamics and equitable partnerships

The project provided opportunities for the participants to debate on everyday questions about equality and to show how these issues are a part of all societal activities. However, to be able to identify and test new approaches (demanded by the financing body), courage and capacity for creative thinking by the involved civil servants and flexible solutions within social systems (i.e. the municipalities where the implementation took place) would have been needed. This was the project’s biggest challenge as the participating municipalities seemed to have a history of using universities and researchers without attention to community engagement. Particularly, considering Roma as equal partners and subjects of transformation processes was an alien concept to the non-Roma civil servants of the target municipalities.

The project, indeed, challenged the prevailing power relations in the communities when the Roma, who earlier had been experienced as in need of help, suddenly were given a voice (Ripamonti et al., 2015) as well as decision-making power in planning and implementing those activities that traditionally had been the municipality authorities’ tasks. The experience of the present project indicates that authorities were rigid and unable to understand the character of an emancipatory approach that included soft values such as critical health literacy and empowerment in social inclusion. Authorities’ unwillingness to adopt innovations is a factor that often jeopardizes the sustainability of an intervention after the grant funding ends (Wallerstein et al., 2008).

About the PAR framework and its ability to improve reflexive practice

The reflexive action research loop was applied as described in the literature (Minkler & Wallerstein, 2008; Reason & Bradbury, 2001; Wallerstein & Duran, 2006). However, it was not easy to distinguish the learning and the consequent action. The learning seemingly produced new meanings (understanding of phenomena) for both the Roma and non-Roma participants, which were tested in the collaborative action in which those meanings were again co-produced (providing new meanings or representations) with the participants of the actions of the project.

The tailor-made training (based on the results of the mapped views and needs of the Roma themselves) generated relevant knowledge (Ripamonti et al., 2015) that obviously was useful in the coordinators’ practice. It also supported them to increase their conceptual understanding for being able to, more effectively and collaboratively, communicate with the higher educated ‘target group’ (civil servants) (see e.g. Puigvert et al., 2012).

About the health literacy and empowerment outcomes

A traditional way of thinking about health literacy includes increasing awareness of the impact of personal behaviours such as smoking, alcohol use, eating habits and physical activity, as well as developing practical skills like reading medicine labels, understanding doctor’s orders and so on. According to Crondahl and Eklund (2012) and Eklund Karlsson et al. (2013), these concepts have less of a value in Roma perceptions of health. For the Roma, health means having access to education and work, having a sense of well-being within the family and having the opportunity to influence their own and the family’s situation and well-being. Achieving this outcome requires increased health literacy about the determinants of health, human rights and societal processes, as well as an ability to use this knowledge to critically analyse their own situations (critical health literacy). Being able to use this knowledge and insight in specific situations characterized the Roma coordinators’ empowerment (Nutbeam, 2008).

In the described project, the participatory approach was meant to include the Roma in the whole process based on their own needs (Wallerstein et al., 2008), an approach that assumedly functioned well for the employed coordinators. We suggest that strengthened health literacy improved the Roma coordinators’ ability to promote health and to increase their opportunities for participation in working life and in decision-making concerning Roma issues. However, the empowerment of the entire Roma community remained marginal due to, e.g. the restrictions and governance by the financing body (see above) and given the too short time span of the project.

In this kind of emancipatory project, quantitative socio-economic evaluations would not necessarily demonstrate added value, as the effects pursued were not economic (Ahl, 2012). Also, the socio-economic effects of the project are more likely to be visible some time (a year or so) after finalizing the project rather than at the end of it (Ahl, 2012; Crondahl & Eklund Karlsson, 2015; Sennemark, 2012). We believe that it is crucial for funding bodies and authorities to look beyond the quantitative data and socio-economic aspects of programmes such as the one described in the present paper. Such ‘hard’ variables clash with ‘soft’ variables like self-confidence, empowerment and dignity.

Conclusions

This paper suggests that health literacy may function as a catalyst in the empowerment process and serve as a tool for analysing and describing the otherwise abstract process of empowerment in more concrete terms. The findings of this study indicate that WIL within a PAR framework enhanced the health literacy and empowerment of the Roma coordinators despite some structural discriminatory obstacles that limited the original bottom-up idea of the project. However, PAR combined with WIL is a suitable research approach for use in work with marginalized people if the researchers’ work is based on the marginalized peoples’ terms and perspectives. Both strategies combine the production of knowledge which is needed for transformation and changes in practice. A prerequisite is that both parties look upon their collaboration as a team and that they participate the work on equal conditions.

This study further shows not only the significance in taking the starting point in the participating groups’ own perspective and needs but also to keep this perspective throughout the whole process. Along with new acquired knowledge, experiences and reflections, and increased level of awareness, the coordinators changed their way of thinking and acting. These changes required flexibility in the structure of the project and the WIL curriculum, allowing for adjustments along the way in accordance to the coordinators’ new insights and needs. This reflexive action research loop is what makes this approach promising in the work for emancipation and social inclusion of marginalized/vulnerable groups. The combination of PAR and WIL opens up for making the genuine needs and perspectives of the participating group central and the essence, through collaboration, equality and reflection.

Finally, the paper suggest more competence development training about empowerment and participatory approaches to civil servants, financing and decision-making bodies in order to better facilitate their potential contribution in transformative processes and social change.

Footnotes

Authors’ contributions

LEK, the principle author of this article, was responsible for the scientific guidance of the study, participation in the data collection and analysis, and revisions of the manuscript. KC was responsible for the action research operations, participation in the data collection and analysis and in writing the article. KCR contributed with critical reflections during the writing process and helped in writing the article. All authors read and approved the final manuscript.

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

Author biographies

Leena Eklund Karlsson is a PhD in Public Health and associate professor at the University of Southern Denmark, Unit for Health Promotion Research. She is also Study Leader of the Bachelor Programmes of Health Education and Promotion and Epidemiology at the Princess Noura University, Riyadh, Saudi Arabia. Her study interest areas are methods for evidence-informed-policymaking, migrant health and health equity.

Karin C Ringsberg is PhD and professor of public health sciences and researcher at the University of Gothenburg, Sweden. She also Dipl. physiotherapist and teacher. Her present central research areas are health and learning from a health literacy perspective, intersectoral work, and global health.

Kristine Crondahl is PhD in Public Health, and senior lecturer at the University West, Department of Social and Behavioural Studies, Sweden. Her research interest areas are empowerment, health promotion of marginalized groups, and health equity.

References

Abel

Bruhin

(2003) Health Literacy/Wissensbasierte Gesundheitskompetenz. In: Bundeszentrale Fur Gesundheitiliche Aufklärung (ed.) Leitbegriffe der Gesundheitsförderung: Glossar Zu Konzepten, Strategien und Methoden in der Gesundheitsförderung. (Guiding principles of health promotion), Schwabenstein a. d. Selz: Peter Sabo, pp. 128–131.

Ahl, R. (2012). Empowerment och delaktighet hos romer i Västsverige. En samhällsekonomisk utvärdering. (Empowerment and participation among Roma people in West Sweden. A socioeconomic evaluation). Unpublished work. Östersund, Sweden: Payoff.

Bates

Thompson

Bates

(2013) Not all dimensions of work self-efficacy are equal: Understanding the role of tertiary work placements in the development of the elements of work self-efficacy. Journal of Cooperative Education and Internships 47(1): 19–30.

Bradbury

Reason

(2003) Action research an opportunity for revitalizing research purpose and practices. Qualitative Social Work 2(2): 155–175.

Choy

Delahaye

(2011) Partnerships between universities and workplaces: Some challenges for work-integrated learning. Studies in Continuing Education 33: 157–172.

Crondahl

Eklund

(2012) Perceptions on health, well-being and quality of life of Balkan Roma adolescents in West Sweden. Romani Studies 22(2): 153–174.

Crondahl, K., & Eklund Karlsson, L. (2015). Roma empowerment and social inclusion through work-integrated learning. SAGE Open, January–March, 1–10.

Crondahl, K., & Eklund Karlsson, L. (2016). The nexus between health literacy and empowerment – A scoping review. SAGE Open, April–June, 1–7.

Eklund, L., & Crondahl, K. (2010). Empowerment och delaktighet hos romer i Väst Sverige. (Empowerment and participation among Roma people in West Sweden). Unpublished final report. Planning, mapping, pre-course and analysis. Gothenburg, Sweden: European Social Fund (ESF).

10.

Eklund Karlsson, L., & Crondahl, K. (2012). Empowerment och delaktighet hos romer i Väst Sverige. (Empowerment and participation among Roma people in West Sweden). Unpublished final report. Implementation. Gothenburg, Sweden: European Social Fund (ESF).

11.

Eklund Karlsson

Crondahl

Sunnemark

Andersson

Å.

(2013) The meaning of health, well-being, and quality of life perceived by Roma people in West Sweden. Societies 3(2): 243–260.

12.

FRA & UNDP. (2012). The situation of Roma in 11 EU Member States. Survey results at a glance. European Union Agency for Fundamental Rights & United Nations Development Programme. Luxembourg, Luxembourg: Publications Office of the European Union. Retrieved from http://fra.europa.eu/sites/default/files/fra_uploads/2099-FRA-2012-Roma-at-a-glance_EN.pdf.

13.

Freire

(1996) Pedagogy of the oppressed, (Rev. ed. New York, NY: The Continuum International Publishing Group Inc.

14.

Freudenberg

Brimble

Cameron

(2010) Where there is a WIL there is a way. Higher Education Research and Development 29(5): 575–588.

15.

Hamel

(2015) Translation between academic research, community and practice: A forum theatre process. The Canadian Journal of Action Research 16: 27–41.

16.

Hancock

(2013) We are the Romani people. Ames am e Rromane dzene, Hatfield, UK: University of Hertfordshire Press.

17.

Hassler, S., & Eklund, L. (2012). Sense of coherence and self-reported health among Roma people in Sweden – A pilot study. International Journal of Circumpolar Health, 71, 18438. doi: 10.3402/ijch.v71i0.18438.

18.

Hodkinson, P., Biesta, G., Gleeson, D., James, D., & Postlethwaite, K. (2005). The heuristic and holistic synthesis of large volumes of qualitative data: The TLC experience. Paper presented at the RCBN 2005 Annual Conference, Cardiff.

19.

Kenny

(2012) An introduction to Moustakas’s heuristic method. Nurse Researcher 19(3): 6–11.

20.

Mårtensson

Hensing

(2012) Health literacy – A heterogeneous phenomenon: A literature. Scandinavian Journal of Caring Sciences 26: 151–160.

21.

Milcher

(2006) Poverty and the determinants of welfare for Roma and other vulnerable groups in Southeastern Europe. Comparative Economic Studies 48(1): 20–35.

22.

Minkler, M., & Wallerstein, N. (Eds). (2008). Community-based participatory research for health: From process to outcomes. San Francisco, CA: Jossey-Bass.

23.

Moustakas

(1990) Heuristic research: Design, methodology, and applications, Thousand Oaks, CA: SAGE.

24.

Murphy

G. A.

Calway

B. A.

(2008) Skilling for the workforce: A tertiary education response to enrich professional development. Tertiary Education and Management 14(2): 95–109.

25.

Nicolae, V. (2012). Roma inclusion: Up to the majority. Eurozine. Retrieved from http://www.eurozine.com/articles/2012-02-24-nicolae-en.html.

26.

Nutbeam

(2000) Health literacy as a public health goal: A challenge for contemporary health education and communication strategies into the 21st century. Health Promotion International 15(3): 259–267.

27.

Nutbeam

(2008) The evolving concept of health literacy. Social Science and Medicine 67(12): 2072–2078.

28.

Palmroth

Hermansson

H.-E.

(2006) Socialt medborgarskap. I & M: Invandrare and Minoriteter 3: 12–15.

29.

Palosuo

(2008) En inventering av forskningen om romer i Sverige. (An inventory of the research on the Roma in Sweden), Uppsala, Sweden: Centre for Multiethnic Research, Uppsala University.

30.

Parry

Van Cleemput

Peters

Walters

Thomas

Cooper

(2007) Health status of gypsies and travellers in England. Journal of Epidemiology and Community Health 61(3): 198–204.

31.

Puigvert

Christou

Holford

(2012) Critical communicative methodology: Including vulnerable voices in research through dialogue. Cambridge Journal of Education 42(4): 513–526.

32.

Purdie

McAdie

King

Ward

(2011) In the right placement at the right time? An investigation of the psychological outcomes of placement learning. Procedia-Social and Behavioral Sciences 29: 717–724.

33.

Rappaport

(1985) The power of empowerment language. Social Policy 16(2): 15–21.

34.

Reason, P., & Bradbury, H. (Eds.). (2001). Handbook of action research: Participative inquiry and practice. London, UK: SAGE.

35.

Regeringskansliet. (2014). Den mörka och okända historien. Vit bok om övergrepp och kränkningar av romer under 1900-talet. (The dark and unknown history. White paper on abuses and violations of Roma during the 1900s). Ds 2014:8. Stockholm, Sweden: Elanders.

36.

Ripamonti, S., Galuppo, L., Gorli, M., Scaratti, G., & Cunliffe, A. L. (2015). Pushing action research toward reflexive practice. Journal of Management Inquiry, 25, 55–68. doi: 1056492615584972.

37.

Sennemark, E. (2012). Kvalitativ Utvärdering av Projekt Empowerment och delaktighet hos romer i Väst Sverige. (Qualitative evaluation of the project empowerment and participation among Roma people in West Sweden). Unpublished document. Gothenburg, Sweden: Contextio Ethnographic.

38.

SOU. (2010). Romers rätt – En strategi för Romer i Sverige. (Roma rights – A strategy for Roma in Sweden). Statens offentliga utredningar, SOU 2010:55. (Official Government Reports) Arbetsmarknadsdepartementet, Delegationen för romska frågor. Stockholm, Sweden. Retrieved from http://www.regeringen.se/sb/d/12482/a/150025.

39.

Springett, J., Wright, M. T., & Roche, B. (2011). Developing quality criteria for participatory health research: An agenda for action. Discussion Paper, Wissenschaftszentrum Berlin für Sozialforschung, Forschungsschwerpunkt Bildung, Arbeit und Lebenschancen, Forschungsgruppe Public Health, No. SP I 2011–302.

40.

Tengland

P.-A.

(2008) Empowerment: A conceptual discussion. Health Care Analysis 16(2): 77–96.

41.

UNDP. (2013). Human Development Report 2013. The rise of the south: Human progress in a diverse world. United Nations Development Programme. New York, NY: UN Plaza. Retrieved from http://hdr.undp.org/sites/default/files/reports/14/hdr2013_en_complete.pdf.

42.

Wallerstein, N. (2006). What is the evidence on effectiveness of empowerment to improve health? Copenhagen, Denmark: WHO Regional Office for Europe (Health Evidence Network report). Retrieved from http://www.euro.who.int/__data/assets/pdf_file/0010/74656/E88086.pdf.

43.

Wallerstein

Duran

(2006) Using community-based participatory research to address health disparities. Health Promotion Practice 7(3): 312–323.

44.

Wallerstein

Oetzel

Duran

Tafoya

Belone

Rae

(2008) What predicts outcomes in CBPR? In: Minkler

Wallerstein

(eds) Community-based participatory research for health. From process to outcomes, San Francisco, CA: Jossey-Bass.

45.

Watts

R. J.

Diemer

M. A.

Voight

A. M.

(2011) Critical consciousness: Current status and future directions. New Directions for Child and Adolescent Development 134: 43–57.

46.

WHO. (1986). Ottawa charter for health promotion. Geneva, Switzerland: World Health Organization. Retrieved from http://www.euro.who.int/__data/assets/pdf_file/0004/129532/Ottawa_Charter.pdf.

47.

WHO (2012) Improving the health of Roma in the WHO European region. An new initiative of the WHO regional office for Europe, Copenhagen, Denmark: World Health Organization, WHO Regional Office for Europe.

48.

Wiggins

(2011) Popular education for health promotion and community empowerment: A review of the literature. Health Promotion International 27(3): 356–371.