‘Missing out’: Reflections on the positioning of ethnographic research within an evaluative framing

Abstract

Contemporary approaches to evaluating ‘complex’ social and health interventions are opening up spaces for methodologies attuned to examining contextual complexities, such as ethnography. Yet the alignment of the two agendas – evaluative and ethnographic – is not necessarily comfortable in practice. I reflect on experiences of conducting ethnographic research alongside a public health evaluation of a community-based initiative in the UK, using the lens of ‘missing out’ to examine intersections between my own ethnographic concerns and those of the communities under study. I examine potential opportunities posed by the discomfort of ‘missing out’, particularly for identifying the processes and spaces of inclusion and exclusion that contributed both to my ethnographic experiences and to the realities of the communities engaging with the initiative. This reveals productive possibilities for a focus on ‘missing out’ as a form of relating for evaluations of the impacts of such initiatives on health and social inequalities.

Keywords

evaluation community empowerment positionality health inequalities inclusion exclusion

Introduction

Amid continuing calls for evaluations of social and health interventions, to understand their mechanisms of effect and to generate ‘evidence’ of their impact (Hawe, 2015), there is increasing recognition of the limitations of experimental approaches to evaluation, resting on probabilistic pathways of causality (Cartwright, 2011), for explaining the successes and failures of ‘complex’ interventions (Blamey and Mackenzie, 2007). This has prompted a re-orientation towards evaluation methods that enable exploration of complexity (Byrne, 2013) and towards understanding the ‘how’, ‘where’ and ‘why’ of the changes produced by a particular intervention – the opening of the ‘black box’ (Stame, 2004) – as opposed to more simply if it works. This opens up space for alternative and ‘non-traditional’ (from a biomedical perspective at least) methodologies such as ethnography to contribute to evaluation of complex interventions (Riley et al., 2005; Cohn et al., 2013).

Ethnography within and/or for evaluation gained prominence in research on educational and social development programmes, particularly from the early 1980s onwards (see for example LeCompte and Goetz, 1982). More recently, alongside increasing recognition of the value of qualitative and mixed methods approaches within health sciences (Pope, 2005), ethnography has been employed alongside experimental trials of health interventions (Savage, 2000a) and has informed evaluations of the health impacts of ‘complex’ interventions (see for example Aronson et al., 2007). Furthermore, ethnography has been prominent in participatory or action-focused approaches to evaluation, seeking to engage both the evaluators and the ‘evaluated’ in collaborative work to develop the effectiveness of a programme (see for example Schensul et al., 1999).

Typically, across this varied literature, two key contributions of ethnography to evaluation practice are highlighted. First is its capacity to attend to ‘context’, or the wider structures of relations and resources that shape how an intervention is delivered in practice, thus shaping the ‘fidelity’ of implementation to the intervention as planned (Dorr-Bremme, 1985; Messac et al., 2013; Bunce et al., 2014). Second, and related, is ethnography’s sensitivity towards capturing multiple and different ‘stories’; the varying experiences and interpretations of those involved (either delivering or receiving the intervention) which may influence the acceptability and/or effectiveness of the intervention (Schensul et al., 1999; Harklau and Norwood, 2005; Riley et al., 2005). Together, these reflect an interpretation of ethnography as being able to fill the ‘gaps’ left by quantitative evaluation methods in explaining how and why an intervention does or does not work. Thus, ethnography’s value for evaluation is framed as lying in its ‘holistic’ perspective (LeCompte and Goetz, 1982; Morgan-Trimmer and Wood, 2016), highlighted particularly in recent systems-based approaches to evaluating ‘complex’ interventions (Aronson et al., 2007; Cohn et al., 2013).

This attention to holism in evaluation literature, however, appears to overlook broader debates across the social sciences in which the evidentiary claims that can be made through ethnography have come under critical reflection. Following the ‘crisis of representation’ within anthropology and related disciplines (Clifford, 1986), it is now commonly acknowledged that ethnographic knowledge is always partial, situated and therefore unstable (Punch, 2012). A wide range of factors have been identified as being inherent to the relational work of ethnographic fieldwork (Coffey, 1999), which necessarily shape and limit what can be known about the ‘object’ of study. These include the identity of the ethnographer in relation to those being studied and the subsequent dynamics of fieldwork, for example relations of mistrust or suspicion, and disconnections arising through ‘foreignness’, as well as relations of connection and good rapport (Hammersley and Atkinson, 2007). They also include dimensions of the hierarchies within the field of study, whereby initial ‘access’ to the field, and ongoing engagement with people, spaces and events, can be determined by ‘gatekeepers’ and the power they hold, and also mediated by the perceived identity of the ethnographer relative to the ‘researched’ (Whitten, 1970; Reeves, 2010). Finally, the very nature of what is being studied and where it is configured poses limitations on a holistic understanding, for example the inevitable ‘incompleteness’ of a multi-sited ethnographic study that seeks to follow ‘flows’ of relations across multiple fields (Hannerz, 2003).

I argue that attention to the partial relating and inevitable disconnections of the ethnographic process must be applied to the increasing engagement with ethnography for evaluation, to consider more critically the claims of holism as its primary contribution. Furthermore, I suggest that conducting ethnography for or with evaluation research may give rise to new forms of relating that shape how ethnographic knowledge is produced, and what it can offer to interpretations of an intervention’s impact on health or social outcomes. In this paper, I engage with relations of disconnection and distance via a concept and experience of ‘missing out’, to reflect on the partial (rather than holistic) processes of knowledge-making through ethnography for evaluation. I focus specifically on the relationship between my ethnographic research and the evaluation agenda, in the context of research on a community-based, empowerment initiative, to identify dimensions of ‘missing out’ that arose from this relationship. I also describe relations of disconnection and distance within the communities under study, and the intersections between our respective ‘missing out’, which I identified following reflection on my methodological practice. Finally, I consider the value of both empirical and methodological dimensions of ‘missing out’ and what they can contribute to evaluations of interventions to address health inequalities.

The ethnographic study

I conducted ethnographic fieldwork over a period of 13 months between 2014 and 2015, to explore enactments of ‘community’ in two areas participating in an area-based (or local community) empowerment initiative being delivered across multiple, relatively disadvantaged areas of the UK. My research question centred on how ‘community’ is enacted through the delivery of the initiative, reflecting prominent discourses of ‘community engagement’ that underpin much contemporary public health practice and social policy-making (Lawson and Kearns, 2010). My research also reflected the often uncritical and largely pragmatic way in which the concept of ‘community’ is operationalized in policies and interventions (Bertotti et al., 2012) and in evaluations of the same, which raises questions about how to interpret the evidence of impact of ‘community’ interventions.

The Initiative

The ‘community’ empowerment initiative at the heart of my research focus was established in the late 2000s, following around a decade or more of UK government-driven policies targeting areas of ‘social exclusion’ deemed to be indicative of, and contributory towards, persisting health and social inequalities in the country (Mackenbach, 2012). Initiatives delivered under these policies have typically involved varying levels of ‘community’ participation (Reynolds et al., 2015), reflecting a range of (predominantly neoliberal) values concerning the rights and responsibilities of citizens in relation to the state (Rose, 2000) and localism agendas, or the devolution of responsibilities for welfare to local arenas (Grover, 2012). Despite a continued emphasis on the importance of engaging the ‘community’, the evidence of the impact of such area-based initiatives on health and social inequalities in the UK (and elsewhere), has been found to be lacking and largely inconclusive (O’Mara-Eves et al., 2013).

The initiative explored in my ethnographic research (hereafter, ‘The Initiative’) sought to adopt an empowerment approach to improving disadvantaged areas across the UK, by situating control for the planning and delivery of The Initiative, and the management of the financial resources allocated to each area, solely in the hands of local ‘communities’. The areas were selected for inclusion in The Initiative by various measures of relative deprivation, and geographic boundaries were stipulated for each area, thus denoting the ‘community’ as the people residing or working within this geographic area (typically between 4000 and 9000 people per area). While The Initiative was ostensibly ‘community’-led, there were various features of its design that structured its delivery in each local site. Each area was facilitated by a centrally-appointed representative to bring together a committee to lead the delivery of The Initiative at the local level. The committees typically comprised between 10 and 15 people, all taking on the role voluntarily, and the majority of whom were expected to be local residents. The committees were tasked with consulting with the wider ‘community’ to identify local priorities for change, and to then develop and deliver a plan to address these over a period of at least 10 years, using financial resources allocated to each area, typically upwards of £1 million. At the time of my research, the two areas included in my fieldwork were in their first few years of participating in The Initiative.

The evaluation study

A programme of public health evaluation research has been developed independently of The Initiative with the aim of exploring the potential impacts on health inequalities, through a theorized pathway of ‘collective control’, a mechanism of collective empowerment arising through the ‘community’s’ control over the delivery of The Initiative. The evaluation study, with its focus specifically on health outcomes, is conducted by academic researchers and funded by a national health research body whereas The Initiative has committed funding from a national charitable organization for 15 years. While the organization delivering The Initiative at the national level has coordinated a range of evaluation activities to assess its progress against its original objectives, the public health evaluation study is being conducted independently of these activities. A first, exploratory phase of research within this public health evaluation study was conducted between 2014 and 2015, using qualitative and quantitative approaches to understand how The Initiative was being rolled out in different local settings, and to identify some of the mechanisms and contextual factors of The Initiative that might contribute to (and be measurable indicators of) ‘collective control’.

My ethnographic study was designed separately from, but in dialogue with, the development of the evaluation study. However, the field sites for my research were two of The Initiative areas that were sampled for the first phase of the evaluation, and therefore my fieldwork was positioned closely alongside the data collection conducted by colleagues from the evaluation study. Consequently, for the purposes of gaining access to the field sites and maintaining relations with residents throughout the research process, I was presented, alongside colleagues, as a member of the evaluation study team conducting various research activities in the area to understand how The Initiative was unfolding ‘on the ground’, but with a specific interest in aspects of ‘community’. The pragmatics of this close positioning of my fieldwork alongside, and occasionally intertwined with, the evaluation study prompted a series of critical reflections on the kinds of ethnographic knowledge I was able to produce through my research and what was absent or missing.

The field sites

The first site, to be known as Westin Hill, is an urban area on the outskirts of a large city, and the boundary of the area, as designated for the purposes of The Initiative, follows exactly the boundary of an electoral ward of the same name, comprising around 12,000 residents. The area of Westin Hill includes a large housing estate along with multiple other streets of residential housing, some small shops and local businesses, and also has a large, well-maintained park that proves a draw for residents and others from outside the area. It is positioned close to a busy high street with good transport links into the city. At the time of my fieldwork, the residents’ committee in Westin Hill were beginning to implement the plan for the local area that had been drawn up following a period of consultation with the ‘community’. The committee comprised around 10 local residents, although this number fluctuated slightly during my fieldwork, and would hold closed meetings every six weeks or so to discuss progress for funding projects and other matters of delivering their plan. Some of the committee members were connected with other local organizations and projects, and a few knew each other via these connections prior to getting involved with The Initiative. The committee also employed a part-time development worker who was responsible for much of the day-to-day administration of delivering The Initiative, although not eligible to vote on decisions made within committee meetings. The committee held larger meetings open to the rest of the ‘community’ every quarter, known as ‘forum meetings’, at which they would feed back on progress against the plan and invite discussion and suggestions for ideas of new projects to fund.

The second site, to be known as Craybourne, is an area within a town near the coast in the north of England. The boundary of Craybourne seems to reflect a locally-recognized neighbourhood, one that is often perceived to be somewhat separate (socially, economically and geographically) from the centre of the town. Within the boundary designated as part of The Initiative, there are around 5000 to 6000 residents, and Craybourne (and the wider town in which it is situated) shows some of the typical signs of a once-popular holiday resort. Craybourne joined The Initiative more recently than Westin Hill, and at the time of starting fieldwork, the committee of residents had only recently begun the ‘community’ consultation process to help identify priorities and develop a plan of action. The residents’ committee was not a fixed group, although it had around 12 local residents who would regularly attend the monthly (open) meetings and had nominated several people into formal positions (chair, vice-chair, secretary, treasurer). A number of committee members were involved with other local organizations, or worked within Craybourne, but a few were fairly new to the area. At the beginning of my fieldwork, the committee appointed a ‘community worker’, employed part-time to lead the consultation process. Other residents, local workers and representatives of local organizations would periodically attend these meetings, though eligibility to vote on decisions made during the meetings was restricted to those who were residents of the designated Westin Hill area.

My fieldwork time was spent moving between the two sites, largely according to the schedule of activities unfolding as part of The Initiative in each area, including committee meetings, events, initiative-funded projects, and other activities occurring locally. As someone who grew up in the north of England (though not particularly near Craybourne), but who now lives in the same city as where Westin Hill is located, my fieldwork engagements with each site were slightly different. For Westin Hill, my approach was that of a ‘commuter’, crossing the city regularly to conduct fieldwork activities. For Craybourne, my time was more periodic but intensive, spending one to two weeks at a time in the area, usually staying in a local Craybourne hotel or with friends nearby. This approach to the timing of fieldwork perhaps reflects what Beaulieu (2010) describes as ‘co-presence’ rather than ‘co-location’ with the research focus, shaped by the ‘when’ rather than the ‘where’ of the field.

Access to both field sites was negotiated, by colleagues from the evaluation study and me, through a series of conversations, first with the organization delivering The Initiative at the national level, then with the appointed representative in each area, and finally with the committee of residents delivering The Initiative in each area, who granted permission for us to conduct our (respective) fieldwork. Consent was sought on an individual basis throughout fieldwork for interviews, and where possible, for observations and informal conversations. As part of these consent processes, we sought to make it clear to residents that the evaluation study and my research were not designed to assess how well they were delivering The Initiative, but to understand more about how The Initiative (and ‘community’) was unfolding in different areas.

Experiencing and interpreting ‘missing out’

Reflection on my research practices occurred throughout my time in the field, often prompting deliberation and concerns over the kind of ethnographic work I was able to do, while positioned in close alignment with the evaluation study. While I acknowledged, from an academic and epistemological sense, the partial nature of my position as researcher and of my interpretations, I still felt ongoing discomfort and uncertainty about the level to which I was able to ‘participate’ in an ethnographic sense in the processes and relations of the ‘community’ delivering The Initiative in each area. This prompted me to consider the kind of experiences, and thus knowledges, I was ‘missing out’ on. Yet, as ethnographers have come to recognize in recent years (Punch, 2012), attending to the emotional dimensions of doing ethnographic research can be a productive, reflexive mechanism for examining the relationships between our experiences and those of our participants (Lee-Treweek, 2000). Considering further my anxieties, I began to recognize parallels between my experiences of ‘missing out’ ethnographically and narratives in my ‘data’ concerning processes and practices of exclusion, distancing and partialness – ‘missing out’ – that intersected enactments of ‘community’ in relation to The Initiative. Here, I will describe the ways in which these ethnographic and empirical accounts of ‘missing out’ came to be identified, and their parallels (if not comparability) interpreted, to demonstrate the potential methodological value of ‘missing out’ for examining the processes and value of knowledge production, alongside an evaluative framing.

The positioning of my ethnographic research in close proximity to the first phase of the evaluation study was one of the first prompts that caused me to reflect on the kind of ethnographic work I was doing, and was able to do, and what I might be missing as a consequence. While the evaluation study followed a largely pragmatic approach to describing the processes of The Initiative within its complex, contextual settings (Hawe et al., 2004), values of a more experimental evaluation approach, aiming (in the future) to establish a (causal) relationship between The Initiative and any measured effects, were also evident. This was implied in attempts to position the work of the evaluation as ‘external’ to, and independent of, that being observed (The Initiative), with the aim of minimizing the influence of the research processes on the mechanisms and effects of The Initiative. This positioning was exemplified in study protocols urging, for example, ‘unobtrusive observation’ of activities, meetings and events during data collection. By nature of my association with the evaluation study, my ethnographic approach was also subject to similar expectations to avoid unduly influencing the people, activities and processes of The Initiative, and thus I felt restricted in my ability to ‘participate’ in the field in ways I would have liked.

Furthermore, there were also implicit restrictions on my capacity to be involved through the structures and values of The Initiative itself which emphasized residents’ control over decision-making for their local area and ‘community’. This posed more moral (as opposed to epistemological) restrictions on the kinds of engagement possible with the field sites through my ethnographic research, and for the evaluation study also. As an ‘outsider’ – someone with no prior connections to either field site, and no intention to live there beyond the fieldwork period – it seemed inappropriate for me to attempt to participate in the decision-making processes that would have real impacts for the local area and its residents in the years to come. Thus, my positioning alongside the evaluation study, and the ideological framing of The Initiative itself, both served to position me, as an ethnographic researcher, at some distance from the focus of my study. My position as a non-resident, and as affiliated with the evaluation study, served to delimit my capacity to engage in a participative way in the field, which prompted reflection on the kinds of ethnographic knowledge I was able to produce.

Missing out ethnographically

So, to turn to the reality of attempting to avoid, where possible, influencing The Initiative, and the people, processes and activities I encountered in my field sites. On a number of occasions during my fieldwork this approach became rather complicated and uncomfortable to negotiate, which subsequently led me to question the kind of ethnographic study I was doing and the knowledge I was producing. Comparing my position with a more traditional, anthropological approach to ethnographic fieldwork, rooted in participant-observation and driven by an ideal of the ethnographer’s full ‘immersion’ into their field (Lewis and Russell, 2011) as the most valid source of knowledge production (Ingold, 2008), I began to feel somewhat lacking and restricted by the need to observe and not participate. The more formal spaces of observation, such as regular committee meetings in which I would sit quietly at the back of the room to take notes, were largely unproblematic. Other scenarios, however, when my presence in relation to residents and their practices was less structured and my role more indistinct, were often more difficult to negotiate. Consequently, I was forced to reflect on what I might have ‘missed out’ on in terms of ethnographic knowledge by not being able to play a participative role akin to the roles played by the people and spaces of my field sites (Ashworth, 1995). Moreover, I was prompted to reflect on what I might be missing, or not fulfilling, in terms of the social expectations constructed through ongoing interactions and relationships with residents.

One such scenario occurred early on in my fieldwork in Craybourne, at a time when I had met only a few of the residents involved in delivering The Initiative, and was still getting to know the area. The residents’ committee had organized a stall along the route of a carnival that would pass through the main town and into the Craybourne neighbourhood, and planned to use the stall to attract members of the public to give their opinions on the priorities for the area, as part of their ‘community’ consultation. Feeling slightly nervous about integrating myself into the group and the activities of the day, I tried to make myself useful on arrival by helping to erect and decorate the stall, and other small tasks. Around these activities, I tried to engage in conversation with the eight or nine members of the residents’ committee who were present and to observe their interactions with members of the public passing by. However, my field notes from the day capture my sense of discomfort at this positioning, as I attempted to be ‘embedded’ within the activities of the day but not be drawn into (inter)actions that might influence the practices or people of The Initiative:

… I was standing behind the stall, trying in most circumstances not to appear too much like I was part of the team – helped by my lack of [Craybourne Initiative] t-shirt – as I didn’t want to be put in the position where I’d have to start talking to some of the members of the public coming up to the stall, as I felt that would be a conflict of interest, and potential impact on the initiative. At times, this felt slightly uncomfortable, when there were people at the stall front wanting to talk but no one free behind the stall, or no one paying attention, to talk to them, and sometimes I would be implicated in the conversations between members of the public and committee members, as I stood next to them, and their gaze would be extended to me, as if to get my agreement or approval on what they were saying … Overall, this was slightly tricky, and perhaps I hadn’t considered fully beforehand what my offer of ‘helping out’ might look like, in terms of trying not to impact on what was going on, but also not to get in the way, and to some degree, to justify my presence and ‘hanging around’… I felt more comfortable keeping myself busy with the more practical activities, such as passing pens, taking in questionnaire sheets, handing stickers etc, than more ‘influential’ activities such as talking to members of the public. (CB-Observation-01, May 2014)

These notes capture something of my assumptions around the kind of ethnography I had anticipated, or assumed, I might do, and the discomfort and challenge of doing this in practice, faced with the perceived constraints of not ‘influencing’ the initiative, either as an ‘outsider’ or as an affiliate to the evaluation study. These notes also hint at the slight sense of discomfort around my precarious position as ‘researcher’ and my perceived need to justify my persistent ‘hanging around’ to those around me. The concept of participant-observation hints at a relationship of (relative) reciprocity with the ‘field’ (Savage, 2000b), but this sense of giving and taking was only ever partial, and often lacking given my positioning in relation to the field, and in relation to the principles of the evaluation agenda.

My sense of ‘missing out’ on part of this assumed reciprocal relationship was also felt at times in Westin Hill, for example in relation to the planning of a cultural festival to be funded as part of the delivery of The Initiative, around eight months into my fieldwork in the area. The planning was largely done in a series of meetings held outside the regular committee meetings, and at the first meeting, Patrick, a committee member leading the planning of the festival, announced that he needed to ‘pick my brains’ regarding food hygiene issues. Suddenly my position as a mere observer appeared to be negated and my potential (or hoped-for) value as a contributor to the planning of the festival was made explicit as Patrick asked me to advise on the environmental health requirements for providing food at the festival. I realized that Patrick had perhaps misinterpreted my background as a public health researcher, and I felt awkward having to correct him and admit that I did not know anything about food hygiene and so could not help. However, later I also reflected on his request as embedded in the developing relationship between me and Patrick (among others in Westin Hill). My inability to respond to his request for my contribution left me feeling uneasy about ‘missing out’ on my side of the reciprocal nature of this relationship, due to my hesitancy over influencing the course of The Initiative. Furthermore, if taking ethnography as a collaborative process (Gottlieb, 1995), it could be suggested that being constrained in my ability to collaborate in a reciprocal sense constrained the depth and value of the ethnographic knowledge produced.

As my time in the field sites progressed I also became aware of particular spaces and forms of interaction arising around the delivery of The Initiative (and of enactments of ‘community’) from which I was disconnected, or of which I was cognisant only from a distance. For example, there were a number of occasions when I would turn up for a committee meeting, or a more informal interaction with committee members, and realize that something had ‘happened’ recently that I had not been aware of. Around five months into my fieldwork in Westin Hill I arrived at a committee meeting and, before the meeting started, I was asked by Nadia, the employed development worker, if I wanted to sign the two greetings cards she had open in front of her. My field notes capture some of my confusion as I tried to understand what had happened in the past couple of weeks:

Nadia seemed to be quite busy with some pieces of paper and other things in front of her and then asked me if I wanted to sign the cards for Colin [a committee member], as his mother had just passed away, and for Jasmine [the committee chair]. I think I was a bit confused about the card for Jasmine and so tried to enquire gently, and Nadia said it was to thank her for all her work now that she’s ‘stepped down’. Nadia was saying this to me as if she thought I knew what had happened, and she was also a bit distracted with other pieces of paper etc in front of her, and chatting with the others, so I tried to ask a few more questions about it, but didn’t get much further than finding out that Jasmine had emailed round after the last committee meeting to say that she was stepping down. (WH-Observation-05, November 2014)

This surprising (but partial) news was an indicator at the time of how I missed out on particular kinds of exchanges and interactions of The Initiative in Westin Hill that occurred particularly in virtual spaces – via email and telephone between committee members and others – from which I was often excluded, likely due to my ineligibility to contribute to the work of the committee, not being a resident of the ‘community’. The context in which I came to find out about Jasmine’s departure from the committee was indicative of the partial nature of my connections with the practices and circulation of knowledge bound up in The Initiative.

There were similar occurrences in Craybourne too, where I would arrive at a meeting and have to pick through pieces of information to work out what had happened or been discussed in spaces that I was not privy to, again by the nature of not being a committee member and fear of disrupting The Initiative. My access to the more informal interactions and ‘goings on’ around the work of The Initiative in Craybourne was fairly variable, however. I was usually included on emails sent around the whole committee, and on a few occasions I was invited, as part of a more select group of residents, to social gatherings at the local pub or at a committee member’s house. At other times, however, I realized I had been missed off certain exchanges, for example, arriving at a committee meeting towards the end of my time in Craybourne to find out that Paul had stepped down unexpectedly from his position as treasurer on the committee following an undisclosed (to me) incident. The chair opened the meeting by asking for a volunteer to take over in Paul’s absence, indicating that although he had done a good job as treasurer, there were ‘very good reasons’ why he was no longer in the role. The language used and explanation given in the meeting by the chair appeared deliberately partial, leading to me writing in my notes that it was a ‘classic case of me feeling like I was missing out’ on key exchanges. I realized that, despite inclusion and connection in some informal and more private spaces of interaction, the distance at which I had positioned myself via (necessary) non-participation meant that my access to the processes and dynamics of the field was not guaranteed or consistent.

Missing out empirically

Yet, I also began to realize that the sense of missing out on certain interactions, knowledge and engagements in the field was not always unique to me as a researcher, restricted in my capacity to participate due to the evaluative framing in which my research was embedded and my status as an outsider. Through more systematic analysis of my empirical ‘data’, I began to identify processes of boundary-making at the heart of the enactment of ‘community’ and the delivery of The Initiative that led to other people, places and things being ‘missed out’. I realized that there were similarities, parallels and occasional intersections between my experiences of ‘missing out’ ethnographically and those of the people and places of my field sites.

Articulations of concern or anxiety over being disconnected from processes of steering The Initiative were fairly prominent among the people with whom I spent much of my time in both field sites. These indicated some individuals’ feelings of ‘missing out’ on their expected role in relation to decision-making in committee meetings, and on behalf of the ‘community’. In both Craybourne and Westin Hill, the appropriateness of particular spaces (physical and virtual) in which conversations and interactions took place that constituted the formal, and more informal, work of progressing The Initiative were, at times, questioned and challenged by members of the residents’ committees (and others) as expressions of perceived exclusion. In Westin Hill, Derek, a dedicated member of the committee, would periodically express his frustration and concern over having ‘missed out’ on email exchanges between other committee members in between meetings, largely due to his apparent reluctance to use a computer.

Several other committee members in Westin Hill indicated, both in meetings and in more informal conversations with me, that they disapproved of others holding smaller, ‘private’ meetings together, for example in a resident’s home. They indicated that the only appropriate place for discussing the work of The Initiative and making decisions was at formal, scheduled committee meetings, typically held at a local community centre. Several times rather heated discussions ensued at committee meetings as Derek and others aired their frustrations, which seemed to indicate both personal concerns over ‘missing out’ on key exchanges, but also their sense of what qualified as ‘appropriate’, non-exclusionary space for the work of representing the ‘community’ in delivering The Initiative. This was an example of the way in which boundaries of what, where and whom did and did not constitute the ‘community’ were continually being drawn and re-drawn through the work of The Initiative.

Narratives of concern about ‘missing out’, relating to representation, also arose around the practices of consulting the ‘community’ on priorities for The Initiative in Craybourne. The processes of planning events and approaches for consulting invariably involved a range of negotiations around the people and spaces still ‘missing’ from their picture of the ‘community’ and its preferences, and discussions about what would be ‘enough’ consultation. For example, in one committee meeting:

Jon asked which groups aren’t engaged … Lucy said specifically the younger and older people, and there was also a suggestion that other groups haven’t been involved like the drug addicts, homeless people and young professionals … There was some discussion about the types of spaces these groups use, particularly services provided by various churches in the area. Lucy said that there’s a need to do this engagement quickly, but the problem is that the group have ‘tended to talk to people who are like us’. There was a suggestion that this is a process that has been done before, but that it needs to be repeated and for the ‘net’ to be cast wider … [Jon said] they need to ask themselves what would be ‘enough’ engagement? Lucy said ‘enough’ would be having more people from the groups they’d identified as missing. (CB-Observation-07, August 2014)

These discussions prompted and intersected a series of practices designed to target the ‘missing’ groups within the ‘community’, including funding Christmas social events at a local arts charity, a community centre and a social club, as a way of accessing different groups and eliciting their opinions on the area. Reporting back from these events at committee meetings prompted some people to reflect on how they had never really known about some of these spaces or the groups of people who attended them, prior to the consultation events, and wondered how many more groups or places they were ‘missing out’.

Thus, the values underpinning The Initiative and the stages that residents’ committees were expected to complete as part of its delivery, including ‘community’ consultation, appear to be influential in shaping narratives of concern about who and why some groups might be ‘missed out’ from representation in the planning of The Initiative. These narratives further intersected notions of accountability of a small sub-group – the committee – towards the broader ‘community’ they represented, and the negotiation of appropriate spaces and mechanisms for progressing the work of The Initiative. They highlighted the processes of segmentation of the collective in attempts to grasp, or approximate the whole ‘community’, and to legitimate decision-making practices, and which contributed to the boundary work of enacting ‘community’ through The Initiative.

Intersections of missing out

The parallels between the two framings of missing out – me, ethnographically, and the field, empirically – were often clear, but it is perhaps helpful to examine a little more closely how to relate the two. In particular, I found that my own points of connection with spaces, activities and people in the two sites would sometimes highlight the relative disconnection of others, and of the processes of The Initiative. At times, this connection/disconnection relation was highlighted through my active pursuit of understanding what lay ‘beyond’ the typical boundaries of The Initiative, and the people and spaces it engaged. As one example, I was prompted to attend a local council meeting in Craybourne, following mention of it at a residents’ committee meeting and the forceful urge of the paid ‘community’ worker to all attendees of the meeting to ‘avoid it at all costs!’ (CB-Observation-04). This strong, active distancing from the work of the local council, despite the apparent relevance of the subject of the meeting (proposed funding for local festivals, with which The Initiative often engaged), provided a compelling reason for me to pursue a tentative connection with this ‘forbidden’ space. Thus, it proved a valuable opportunity for me to explore the role of ‘boundary work’ in enactments of ‘community’, and how the work of delivering The Initiative contributed to the ongoing assertion and negotiation of boundaries between different spaces, people and organizations, for different purposes.

A more complex scenario of (dis)connection also arose in Craybourne, about mid-way through my fieldwork at a time when there were some tensions among members of the committee, and meetings were occasionally disrupted by arguments over the progress and direction of The Initiative. One afternoon I met Angela, a committee member, for coffee and a chat, and she told me at length about her upset at a complaint made against her in a public forum by Frank, another committee member, and someone she had known quite well prior to The Initiative. She felt frustrated that she had not been given an opportunity to respond to his complaint, and seemed saddened by the disruption she felt his complaint had made for the committee trying to get on with the work of The Initiative. I realized that I had been missed off the original set of emails in which the complaint had been shared, and so was surprised a week or so later when I discovered Angela had ‘blind copied’ me into an email from her to Frank and several others connected with The Initiative. In this email she responded to Frank’s complaint and outlined her own complaint against what she described as his ‘bullying’ behaviour. This partial ‘inclusion’ of me on a rather heated, contentious set of interactions prompted reflections on my ambiguous positioning in relation to the issue:

After I realised I’d been Bcc’d into the email, I felt slightly uncomfortable, feeling somehow conspiratorially implicated in Angela’s plans to complain, and uneasy that I was being privy to some information that others in the committee wouldn’t know was being shared, and perhaps wouldn’t want me to see. (CB-Fieldnote-19; November 2014)

Here I was simultaneously connected and kept at a distance by the secretive way in which I had been granted access to the exchange and, as a result, I felt I was unable to discuss this issue with anyone other than Angela, and subsequently ‘missed out’ on the emails in response to her complaint.

Similarly, in Westin Hill, my own developing ethnographic engagements prompted identification of other points of disconnection or missing out. An example of this occurred around one of the first programmes funded through The Initiative as part of delivering their approved plan, which was a sports programme for young people, held in a communal space at the main housing estate in Westin Hill. I attended one of the programme’s four sessions, to observe The Initiative ‘in action’ and to chat with local residents and young participants, and, unexpectedly, was encouraged to participate by the enthusiastic organizers. I was surprised, however, that from the committee only Nadia, the development worker, was present when I attended and, from her indication, only one other committee member had come along to watch the sessions. At a committee meeting a few months later, the disconnection of the committee from this activity was further emphasized when they discussed how to increase their engagement with young people in the area, and one member reflected that perhaps they should have attended the programme sessions as a way of connecting with young people. This highlighted the shifting ways in which my points of connection, in the form of attending particular spaces and engaging with people and activities of The Initiative (and of the ‘community’), brought into view perceptions of the disconnection of others, expressed particularly through their concerns of ‘missing out’ on the involvement of specific groups of people, such as young people. It also indicated ways in which the ‘community’ as a collective was frequently segmented, physically and conceptually, into priority spaces and groups, through the delivery of The Initiative.

Hence, reflection on the emotional dimensions of my sense of ‘missing out’ on important aspects of the production of ethnographic knowledge served to highlight both ethnographic and empirical accounts of disconnection, and their intersection, in relation to the enactment of ‘community’ through The Initiative. Thus, my sense of ‘missing out’ appeared not only to reflect the constraints posed by my methodological positioning in relation to an evaluation agenda, but also my position relative to the complex series of connections and disconnections arising among different sets of actors, spaces and processes in my field sites. As Mesman argues in her account of ongoing positioning of the researcher in ethnographic fieldwork, the disruptions we experience in our research both highlight and reflect the complex ‘dynamics of the local-interactional spaces’ (2007: 293) that we seek to understand. It was through attending to these relations of ‘missing out’ that I was further able to identify the processes of ongoing ‘boundary work’ (construction and negotiation of boundaries), and processes of segmenting ‘the collective’, as fundamental components of the enactment of ‘community’ through The Initiative.

Conclusions

The notion of ‘missing out’ has been explored as both an emotionally-rooted experience of engaging (or otherwise) with the social and spatial relations of the ‘object’ of my research – enactments of ‘community’ in relation to an area-based initiative – and an analytical construct relating to processes of boundary making underpinning the delivery of The Initiative for the ‘community’ in these contexts. The role played by ‘missing out’ in linking the methodological reality of ethnographic research and the empirical reality of delivering The Initiative reflects a mutuality between modes of knowing and being in the field, and between interacting with, and interpreting, the ‘object’ of study (Ingold, 2008). For Pigg, the continual critical reflection that is inherent to ethnography means that knowledge production necessarily intersects the ongoing process of ‘everyone’s efforts to make sense of what is going on’ (2013: 132) in the field, researcher and researched alike. Following Punch (2012) and Lee-Treweek (2000), the emotional dimensions of feeling I was ‘missing out’ ethnographically, due to my alignment with an evaluation agenda, served as a productive form of reflexivity that enabled me to consider the relationship between my experience of the ‘community’ around The Initiative in each field site and the experiences of the residents themselves. So, while my anxieties about not being able to participate in the practices of The Initiative and the ‘communities’ delivering it reflected a form of disconnection from the activities in the field, this could also be seen as a kind of participation in the processes of exclusion and boundary-making that permeated enactments of The Initiative.

Hastrup (2005), following Strathern (1995), argues that ethnographic fieldwork offers a particular mode of perception ‘consisting of, and steeped in, social relations’ (2005: 141), and that it reflects the distinctive characteristic of anthropology that assumes that knowledge is created through such relations. It is possible to theorize ‘missing out’ as a form, rather than a lack, of social relation; a type of (dis)connectivity to a socio-spatial sphere through awareness of its presence and through perception of one’s position outside of it or excluded from it. Thus ‘missing out’ can potentially be viewed as a valid form of ethnographic knowledge production, such that it reflects and arises out of particular sets of embedded relationships with the people and places of the field. There are multiple factors that shape the relationship and positioning of the ethnographer with the field, including dimensions of identity, ‘access’ and rapport (Hammersley and Atkinson, 2007). Here, however, I have illustrated how an attempt to align ethnography with an evaluation framing poses particular (additional) forms of ‘missing out’, and thus prompts new ways of considering the contribution of ethnographic knowledge to evaluating the impacts of an intervention such as The Initiative.

This perspective holds parallels with a range of anthropological and sociological literatures on absences, concealments and disconnections in the processes of ‘knowing’. Reflecting currents from Science and Technology Studies in particular, there are various anthropological interpretations of the work that secrets and concealments do in the processes and relations of knowledge production, for example in the context of international diplomatic deliberations (Rappert, 2010) or in clinical trials (Geissler, 2013). Such accounts convey theorizations of ‘unknowing’, and the absences, disconnections and concealments underpinning ‘unknowing’, not as the dialectical opposite of knowledge, but as part of the economies of social and political relations that construct boundaries around what counts as ‘knowledge’, and by and for whom (Dilley, 2010; Rappert, 2010; Geissler, 2013). Thus, the fact that I ‘missed out’ on engagement with certain practices and contexts, such as participating in the delivery of The Initiative, does not necessarily equate with a subsequent lack of (ethnographic) knowledge.

Considering also my position as ethnographer in relation to the people, practices and spaces of my research ‘object’, it is useful to think about ‘missing out’ not merely as an absence of social closeness or connection, which would denote a simplistic subject/object division, but as part of a more fluid and pluralistic way of relating. Alternative ways of interpreting the positioning of the researcher allow for disconnection and distance as productive types of social relations, for example Candea’s identification of a ‘middle ground between intersubjectivity and the absence of relation’ in scientific relations (2010: 249), and Coleman’s description of ‘collective solitude’ as a form of sociality in busy, urban spaces (2009). These examples suggest that my experiences of disconnection and distance need not be cast solely as regrettable ‘limitations’ of my ethnographic methodology, treating them as ‘missing data’ in a traditional scientific way. Rather, the experiences of ‘missing out’, as legitimate forms of relating and positioning in the field, can be acknowledged as prominent and productive components of the construction of knowledge (Dilley, 2010).

I am cautious, however, about suggesting that my experiences of ‘missing out’ as a researcher were wholly commensurate with those experiences of the people and places of my research, in the enactments of ‘community’ around The Initiative. The forms of relating I have identified here reflect largely the restrictions posed by the agenda of conducting evaluation research, underpinned by authoritative forms of ‘scientific’ knowledge (Mykhalovskiy et al., 2008) that did not intersect residents’ own concerns about who or what might be ‘missing’ from the process of delivering The Initiative. However, there were much closer ties between the localized moral economies underpinning the processes of residents working for, and representing, the ‘community’ in delivering The Initiative and my own sense of (self-)exclusion from participating as an ‘outsider’, in terms of who was eligible to contribute and in which spaces this could legitimately happen.

Implications for evaluation

Given the entanglement of my ethnographic knowledge production with the values of evaluation research, it is useful to consider what my reflections on ‘missing out’ might offer to the evaluative agenda. I suggest that ‘missing out’ offers a conceptual means to consider how evaluation approaches might attend more closely to the intersection of the methodological and empirical. This may shed light on the spaces, processes and relations that may be unknown and/or unknowable in research seeking to understand how an intervention works (or does not), and the impacts it has, and for whom. Contemporary evaluation approaches, in seeking to ‘unpack the black box’ (Stame, 2004), assume a holistic epistemological perspective, though typically with a critical pragmatism that acknowledges the limitations of research methods. Yet, they often attempt to negate, reduce or explain away the uncertainties, absences and discontinuities that arise through the research process, reflecting the need to demonstrate the validity of research and fulfil the requirements of ‘good research practice’ (Reynolds et al., 2014).

Turning attention to the role that absences and relations of disconnection play in shaping understanding and interpretations of, for example, a community empowerment initiative, must surely contribute to the agenda to uncover ‘hidden’ processes of change that previous, experimental forms of evaluative research could not reveal. While it may be a fallacy to assume that by generating more knowledge we will have more control (Ling, 2012), it is possible that by better understanding the limits of our knowledge – identifying the spaces occupied by the ‘known unknowns’ (Rappert, 2010) – the evaluative agenda of generalizing evidence of how an intervention brings about change could be improved through more precise circumscription.

Attending more closely to processes of disconnection and ‘missing out’ must surely be vital for public health evaluations in particular, facilitating, for example, interpretation of the mechanisms of an intervention such as The Initiative and how they shape local access to resources that influence health (and inequalities) for different people (Gatrell, 1997). Attention to ‘missing out’ may also offer a more critical interpretation of the mechanisms through which collective action plays out in context, for example in a community empowerment initiative which assumes increasing solidarity and mutual support will lead to improved psycho-social wellbeing (Campbell and Murray, 2004). This perspective may reflect a similar ‘bias of solidarity’ of which ethnography has been accused (Coleman, 2009), which means evaluations may overlook the subtle and fluid roles that disconnections, absences and distancing seem to play among the people and places of an intervention.

Despite my proposition that ‘missing out’ methodologically should not be merely explained away or bracketed as a regrettable ‘limitation’, it is important to consider the possible limitations of the approach I have taken to identifying ‘missing out’ as a valuable conceptual and analytical tool. My experiences of ‘missing out’ methodologically could be interpreted as indicators of where I could have tried harder, gone further, or stayed longer in my ethnographic approach to make more, deeper, or ‘better’ connections with the objects of my study. Certainly, my movement between the two field sites likely contributed to the ‘incompleteness’ of my interpretations (Hannerz, 2003). It could also indicate the unsuitability of a traditional ethnographic methodology for exploring an initiative that limited my participation, or that my assumptions about what ethnography would entail were misguided. However, I have identified that through my ‘missing out’ I was able to engage in a form of ‘participation’ that contributed to my readings of the social reality of an initiative-in-context and which may enhance the evaluative interpretation of how and for whom it brings about change.

These findings also point to outstanding questions that cannot be explored fully here, such as how my interpretations of the ‘community’ in my field sites were likely influenced by other dimensions of relating, including perceptions of identity and related rapport-building. My insights also highlight something of the complex ‘folding together’ of the methodological, empirical and ethical dimensions of ethnographic research (Parker, 2007), via the relations of connection and disconnection with research participants. Thus, there remain questions of how ‘missing out’ on the reciprocal nature of participation intersects expectations for the relations and exchange between ethnographer and participants in, and beyond, the field (Scheper-Hughes, 2001).

To conclude: The value of ethnographic work for evaluation has often been centred on its offer of more holistic understanding of how an intervention works in context (LeCompte and Goetz, 1982). However, I propose that an ethnographic approach can play a different, and potentially more valuable, role in bringing to light the spaces occupied by the unsaid, the unrepresented and the parts of the social ‘whole’ that are characterized by disconnection, absence and partialness. I argue that within evaluation research, ethnographic attention to ‘missing out’, and how it manifests methodologically as well as empirically, will help to highlight the processes through which spaces, people and things become excluded, distanced or disconnected, and which may be obscured in standard evaluation approaches resting on models of causality. This will potentially add depth, as well as clarity of limitations, on interpretations of how change might be produced through a community initiative, and by whom its effects (health-related or otherwise) will and will not be experienced.

Footnotes

Acknowledgements

I wish to acknowledge the inputs and guidance of Mark Petticrew, Matt Egan and Simon Cohn (all LSHTM), and the support of colleagues from the School for Public Health Research conducting the evaluation study described in this paper.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by a grant awarded by the School for Public Health Research at LSHTM, funded through the NIHR School for Public Health Research. The views expressed are those of the author and not necessarily those of the NHS, the NIHR or the Department of Health.

Author Biography

Joanna Reynolds is completing a PhD in Public Health with the NIHR-funded School for Public Health Research at London School of Hygiene & Tropical Medicine. She has a BA (Hons) in Social Anthropology from University of Cambridge and an MSc in Public Health from LSHTM. She has worked previously as a Research Fellow conducting qualitative research to explore dimensions of evaluating complex interventions to improve the diagnosis and treatment of malaria.

References

Aronson

Wallis

O’Campo

Whitehead

Schafer

(2007) Ethnographically informed community evaluation: A framework and approach for evaluating community-based initiatives. Maternal and Child Health Journal 11(2): 97–109.

Ashworth

(1995) The meaning of ‘participation’ in participant observation. Qualitative Health Research 5(3): 366–387.

Beaulieu

(2010) From co-location to co-presence: Shifts in the use of ethnography for the study of knowledge. Social Studies of Science 40(3): 453–470.

Bertotti M, Jamal F and Harden A (2012) Connected Communities: A Review of Conceptualisations and Meanings of ‘Community’ Within and Across Research Traditions: A Meta-Narrative Approach. London, UK, Arts and Humanities Research Council. Available from: http://www.ahrc.ac.uk/documents/project-reports-and-reviews/connected-communities/a-review-of-conceptualisations-and-meanings-of-community-within-and-across-research-traditions-a-metanarrative-approach (accessed 31 May 2016).

Blamey

Mackenzie

(2007) Theories of change and realistic evaluation: Peas in a pod or apples and oranges? Evaluation 13(4): 439–455.

Bunce

Gold

Davis

McMullen

Jaworski

Mercer

et al. (2014) Ethnographic process evaluation in primary care: Explaining the complexity of implementation. BMC Health Services Research 14(1): 1–10.

Byrne

(2013) Evaluating complex social interventions in a complex world. Evaluation 19(3): 217–228.

Campbell

Murray

(2004) Community health psychology: Promoting analysis and action for social change. Journal of Health Psychology 9(2): 187–195.

Candea

(2010) ‘I fell in love with Carlos the meerkat’: Engagement and detachment in human-animal relations. American Ethnologist 37(2): 241–258.

10.

Cartwright

(2011) A philosopher’s view of the long road from RCTs to effectiveness. The Lancet 377(9775): 1400–1401.

11.

Clifford

(1986) Introduction: Partial truths. In: Clifford

Marcus

(eds) Writing Culture: The Poetics and Politics of Ethnography, Berkeley, CA: University of California Press, pp. 1–26.

12.

Coffey

(1999) The Ethnographic Self: Fieldwork and the Representation of Identity, London: Sage.

13.

Cohn

Clinch

Bunn

Stronge

(2013) Entangled complexity: Why complex interventions are just not complicated enough. Journal of Health Services Research & Policy 18(1): 40–43.

14.

Coleman

(2009) Being alone together: From solidarity to solitude in urban anthropology. Anthropological Quarterly 82(3): 755–777.

15.

Dilley

(2010) Reflections on knowledge practices and the problem of ignorance. Journal of the Royal Anthropological Institute 16: S176–S192.

16.

Dorr-Bremme

(1985) Ethnographic evaluation: A theory and method. Educational Evaluation and Policy Analysis 7(1): 65–83.

17.

Gatrell

(1997) Structures of geographical and social space and their consequences for human health. Geografiska Annaler: Series B, Human Geography 79(3): 141–154.

18.

Geissler

(2013) Public secrets in public health: Knowing not to know while making scientific knowledge. American Ethnologist 40(1): 13–34.

19.

Gottlieb

(1995) Beyond the lonely anthropologist: Collaboration in research and writing. American Anthropologist 97(1): 21–26.

20.

Grover

(2012) Localism and poverty in the United Kingdom: The case of Local Welfare Assistance. Policy Studies 33(4): 349–365.

21.

Hammersley

Atkinson

(2007) Ethnography: Principles in Practice, 3rd ed. London: Routledge.

22.

Hannerz

(2003) Being there … and there … and there! Reflections on multi-site ethnography. Ethnography 4(2): 201–216.

23.

Harklau

Norwood

(2005) Negotiating researcher roles in ethnographic program evaluation. Anthropology & Education Quarterly 36(3): 278–288.

24.

Hastrup K (2005) Social anthropology. Towards pragmatic enlightenment? Social Anthropology 13(2): 133–149.

25.

Hawe

(2015) Lessons from complex interventions to improve health. Annual Review of Public Health 36(1): 307–323.

26.

Hawe

Shiell

Riley

(2004) Complex interventions: How ‘out of control’ can a randomised controlled trial be? British Medical Journal 328(7455): 1561–1563.

27.

Ingold

(2008) Anthropology is not ethnography: Radcliffe-Brown lecture in social anthropology, 2007. Proceedings of the British Academy 154: 69–92.

28.

Lawson

Kearns

(2010) Community engagement in regeneration: Are we getting the point? Journal of Housing and the Built Environment 25(1): 19–36.

29.

LeCompte

Goetz

(1982) Ethnographic data collection in evaluation research. Educational Evaluation and Policy Analysis 4(3): 387–400.

30.

Lee-Treweek

(2000) The insight of emotional danger: Research experience in a home for older people. In: Lee-Treweek

Linkogle

(eds) Danger in the Field: Risk and Ethics in Social Research, London: Routledge, pp. 114–131.

31.

Lewis

Russell

(2011) Being embedded: A way forward for ethnographic research. Ethnography 12(3): 398–416.

32.

Ling

(2012) Evaluating complex and unfolding interventions in real time. Evaluation 18(1): 79–91.

33.

Mackenbach

(2012) The persistence of health inequalities in modern welfare states: The explanation of a paradox. Social Science & Medicine 75(4): 761–769.

34.

Mesman

(2007) Disturbing observations as a basis for collaborative research. Science as Culture 16(3): 281–295.

35.

Messac

Ciccarone

Draine

Bourgois

(2013) The good-enough science-and-politics of anthropological collaboration with evidence-based clinical research: Four ethnographic case studies. Social Science & Medicine 99: 176–186.

36.

Morgan-Trimmer

Wood

(2016) Ethnographic methods for process evaluations of complex health behaviour interventions. Trials 17: 232.

37.

Mykhalovskiy

Armstrong

Bourgeault

Choiniere

Lexchin

et al. (2008) Qualitative research and the politics of knowledge in an age of evidence: Developing a research-based practice of immanent critique. Social Science & Medicine 67(1): 195–203.

38.

O’Mara-Eves

Brunton

McDaid

Oliver

Kavanagh

Jamal

et al. (2013) Community engagement to reduce inequalities in health: A systematic review, meta-analysis and economic analysis. Public Health Research 1(4): DOI: http://dx.doi.org/10.3310/phr01040.

39.

Parker

(2007) Ethnography/ethics. Social Science & Medicine 65(11): 2248–2259.

40.

Pigg

(2013) On sitting and doing: Ethnography as action in global health. Social Science & Medicine 99: 127–134.

41.

Pope

(2005) Conducting ethnography in medical settings. Medical Education 39(12): 1180–1187.

42.

Punch

(2012) Hidden struggles of fieldwork: Exploring the role and use of field diaries. Emotion, Space and Society 5(2): 86–93.

43.

Rappert

(2010) Revealing and concealing secrets in research: The potential for the absent. Qualitative Research 10(5): 571–587.

44.

Reeves

(2010) A difficult negotiation: Fieldwork relations with gatekeepers. Qualitative Research 10(3): 315–331.

45.

Reynolds J, DiLiberto D, Mangham-Jefferies L, Ansah E, Lal S, Mbakilwa H, et al. (2014) The practice of ‘doing’ evaluation: Lessons learned from nine complex intervention trials in action. Implementation Science 9(1): 75.

46.

Reynolds, J, Egan M, Renedo A, et al. (2015) Conceptualising the ‘community’ as a recipient of money – A critical literature review, and implications for health and inequalities. Social Science & Medicine 143: 88–97.

47.

Riley

Hawe

Shiell

(2005) Contested ground: How should qualitative evidence inform the conduct of a community intervention trial? Journal of Health Services Research and Policy 10(2): 103–110.

48.

Rose

(2000) Community, citizenship, and the Third Way. American Behavioral Scientist 43(9): 1395–1411.

49.

Savage

(2000a) Ethnography and health care. British Medical Journal 321: 1400–1402.

50.

Savage

(2000b) Participative observation: Standing in the shoes of others? Qualitative Health Research 10(3): 324–339.

51.

Schensul

LeCompte

Hess

Nastasi

Berg

Williamson

et al. (1999) Using Ethnographic Data. Interventions, Public Programming and Public Policy. Ethnographer’s Toolkit: Book 7, Walnut Creek, CA: Alta-Mira Press.

52.

Scheper-Hughes

(2001) Saints, Scholars and Schizophrenics: Mental Illness in Rural Ireland. Twentieth Anniversary Edition, Berkeley, CA: University of California Press.

53.

Stame

(2004) Theory-based evaluation and types of complexity. Evaluation 10(1): 58–76.

54.

Strathern M (1995) The Relation. Issues in Complexity and Scale. Cambridge, UK: Prickly Pear Press.

55.

Whitten

(1970) Network analysis and processes of adaptation among Ecuadorian and Nova Scotian negroes. In: Freilich

(ed.) Marginal Natives: Anthropologists at Work, New York, NY: Harper and Row, pp. 339–402.