On the Brink of Genuinely Collaborative Care

What Is Experience-Based Co-Design?

EBCD is a collaborative approach that aims to improve health care services by enabling service-users, carers, and staff (ground-level and management) to collaborate together to co-design better services. Originally known as experience-based design (EBD), Bate and Robert (2006) argued that adding the “co” emphasizes the partnership of different groups working together. The approach draws on participatory action research, user-centered design, learning theory, and narrative-based approaches to change (Robert, 2013). It was first piloted in a Head and Neck Cancer service (Bate & Robert, 2007), and subsequently a toolkit was developed (The King’s Fund, 2012). The sequence followed by EBCD projects has evolved with the literature, but tends to follow a basic process of (a) gathering experiences from staff, then service-users and carers, via observation and interviews (which are often filmed); (b) identifying “touchpoints” (critical moments experienced in relation to the service) and (c) feeding these back to the project participants; (d) prioritizing the touchpoints by the project participants; (e) bringing everyone together in a co-design event, where they work in small groups to co-design improvements to the service according to the priorities identified; and (f) holding a celebration event to allow all involved to review what has been achieved (Donetto, Tsianakas, & Robert, 2014; Robert, 2013).

What Do We Know About EBCD in Practice, and How Can We Evaluate It?

EBCD has been used relatively widely in U.K. health care and championed by the NHS Institute for Innovation and Improvement (2013; recently replaced by NHS Improving Quality) and by the health care charity, The King’s Fund (2012). A recent report on the use of EBCD identified 59 studies in six countries, the majority of which were in physical health settings (Donetto et al., 2014). Of these, only a minority have published their findings independently. These include the original Head and Neck Cancer care project (Bate & Robert, 2007), projects in outpatient services for older adults (Bowen et al., 2013), breast cancer care (Boyd, McKernon, Mullin, & Old, 2012), compassionate care in hospital (Dewar, Mackay, Smith, Pullin, & Tocher, 2010), wound care (Grocott, Blackwell, Currie, Pillay, & Robert, 2012), emergency care (Idema et al., 2010), breast and cancer care (Tsianakas et al., 2012), and stroke services (Tunny & Ryan, 2014). However, the use of EBCD in mental health services is still relatively novel, and we are only aware of two other NHS services that have implemented EBCD in acute mental health care and one that has used it in community services for young people.

One challenge when service-users and carers are involved in shaping healthcare is understanding the impact of their involvement on services; this has not been well documented to date (Mockford et al., 2012). About 70% of EBCD studies surveyed by Donetto et al. (2014) included evaluation; however, these were most frequently written up in the form of an internal report, and only a handful of evaluations are publicly available: Piper, Iedema, and Merrick (2010) and Piper et al., (2012) evaluated an EBCD project aimed at improving emergency departments in New South Wales, Australia; The King’s Fund (2011) evaluated an EBCD project in breast cancer services with staff and patients; and Bowen and colleagues (Bowen, Dearden, Wolstenholme, Cobb, & Wright, 2010; Bowen et al., 2013) evaluated their use of EBCD to improve older adults outpatient services in the United Kingdom. There are a number of similarities between these reports, although it is important to note that one evaluation only included the staff participants (Piper et al., 2010).

All evaluations reported that some of the co-design improvements were successfully implemented. Piper et al. (2012, p. 167) noted that a strength of the EBCD approach was its “ability to bring about improvements in both the operational efficiency and the inter-personal dynamics of care at the same time,” and all evaluations mentioned secondary benefits such as better communication (The King’s Fund, 2011; Piper et al., 2010), stronger staff team relationships (The King’s Fund, 2011; Piper et al., 2012; Piper et al., 2010), and participants feeling “listened to,” consulted, and involved (Bowen et al., 2010; Bowen et al., 2013, Piper et al., 2010). Piper and colleagues (Piper et al., 2012; Piper et al., 2010) noted that staff valued the opportunity to understand others’ perspectives more intimately and appreciated the opportunity to work together meaningfully to produce important changes. Inevitably, there were also a number of challenges reported, many of which are common to any change management process (Piper et al., 2012). These included issues with governance and the attainability and sustainability of improvements (Bowen et al., 2010; Bowen et al., 2013; The King’s Fund, 2011; Piper et al., 2012).

What has been published about EBCD suggests that it can be a powerful mechanism for service improvement, making services more acceptable to service-users, carers, and staff, thus potentially increasing well-being for all concerned. However, the challenges encountered do illustrate the importance of implementing the approach with the right support and resources. We believe there are also some important distinctions between physical and mental health, which mean that there are likely to be additional challenges in mental health settings. These differences imply that the EBCD approach needs further refinement to be used successfully and safely in a mental health context. For example, ethical issues are likely to be of particular concern when using EBCD in mental health, and these are understated in the EBCD literature. Issues regarding the potential for (re)traumatization, consent (particularly with regard to the legacy of film recordings), anonymity, confidentiality, power, and data “ownership” all require further exploration. Services users may be vulnerable, have histories of trauma and abuse, or may experience difficulties that make them concerned about being recorded (such as paranoia). In addition, risk to self and others is more prevalent than in physical health care settings, and, if identified during the research, will need careful management. Methodological and ethical rigor, consistent with formal qualitative research, are required to ensure that EBCD is being used to its full potential and in a way that is safe and appropriate. In the next section we describe our adaptation of EBCD, and how we engaged with these issues.

Findings From Research Phase

The research phase was granted NHS Research Ethics Committee approval, which included gaining consent for anonymized interview extracts to be shared in presentations and publications. The research involved audio recording in-depth one-to-one interviews with purposively sampled service-users (N = 6), parents (N = 6), and inpatient staff (N = 9). The transcripts from these interviews were analyzed using interpretative phenomenological analysis (IPA; Smith, Flowers, & Larkin, 2009). Findings (reported in detail elsewhere) suggest that hospitalization in early psychosis could be difficult for service-users, their families, and the inpatient staff. The young service-users described their experiences as frightening and confusing (Fenton et al., 2014) whereas staff felt frustrated that the inpatient environment was not more therapeutic, that they “lurched from crisis to crisis,” and that they were not able to spend more time interacting with young service-users (Thompson et al., 2015). Parents were also unhappy, and although they emphasized the necessity of the hospitalization, they felt poorly informed, blamed, and excluded from this critical stage in the young person’s care (Hickman et al., 2015).

Together, these findings indicate a pattern of “shared uncertainty” around experiences of hospitalization in early psychosis, and implied that improvements were needed to shift service-users’ experience from passive seclusion in hospital to active preparation for discharge and recovery; shift staff’s experience from managing crises to professional satisfaction; and shift parents’ experience from feeling disregarded to feeling involved. It is likely that similar improvements are needed across many inpatient mental health services (see Mind recommendations [Mind, 2011]; NICE guidance [NICE, 2011b] and standards [NICE, 2011a]; Rethink recommendations [Rethink, 2010]; The Royal College of Psychiatrists standards [The Royal College of Psychiatrists, 2011] and Accreditation for Inpatient Mental Health Services (AIMS) guidance [AIMS, 2010]).

Converting the Research Phase Into “Experience-Based Co-Design”

The research allowed service-users, parents, and staff to express their concerns and experiences (both positive and negative) about hospitalization for early psychosis. We presented our research team’s findings to senior colleagues at the service, who were keen to use the research as a catalyst for developing inpatient care to better meet the needs of young people with psychosis. Building on our previous experience of using EBCD in physical health care (Taylor, Dhinse, & Larkin, 2012), we believed that EBCD was a good candidate for translating research findings into service improvements, in a way that would be collaborative and inclusive.

To begin translating experience into co-design, the complex thematic findings were converted into a more accessible and concrete list of “touchpoints.” A touchpoint is any crucial moment that makes a difference (good or bad) to someone’s experience of the environment or process (Robert, 2013). In traditional EBCD, Robert (2013) recommends that two qualitative researchers watch the filmed interviews to identify a set of touchpoints. In the current project, touchpoints were identified (see Table 2) by refining and condensing the qualitative analysis of the interviews.

OPEN IN VIEWER

Table 2.

The Touchpoints.

1	Hospital is psychologically containing; a safe place to be
2	Hospital provides respite for service-users and families
3	Wards feel unsafe/frightening
4	Wards are unpredictable/chaotic
5	Wards are non-therapeutic environments
6	Opportunities for service-users to learn from each other
7	Lack of formal support, clinical supervision, debriefing for staff
8	Inadequate support for carers and families
9	Service-users feel cared for by staff
10	Role of relatives and friends in promoting recovery
11	Links with Early Intervention, Crisis teams, and other organizations
12	Not enough therapeutic work, not enough staff-patient contact
13	Lack of resources
14	Lack of explanation / information for families and patients
15	Families feel excluded
16	Feeling unprepared for discharge
17	Dealing with incidents
18	Positive motivations to work with younger patients and make a difference

Feedback Groups

Having identified the touchpoints, we facilitated 20 “feedback groups,” consisting separately of inpatient staff, community mental health staff, NHS managers, family members, or service-users. At each group, the findings from the research projects were presented, and attendees were invited to choose their “priorities for change.” This allowed the views of over 150 stakeholders to be heard. Consensus was reached by first dividing each feedback group into smaller groups of two to three people. Each of these groups were provided with the touchpoints and some example data extracts, and asked to cluster together any touchpoints felt to “belong” together (e.g., different aspects of same problem), to give names to any new clusters they created, and to make a record of which touchpoints were included. We then asked the groups to list their top five priorities (where any one priority could be a single touchpoint or a cluster of touchpoints), making a note of why these had been chosen. After all the 20 feedback groups had met, priorities were tabulated and a thematic approach was taken to identify the most prevalent. For example, there were 17 mentions of the ward environment, so this became one priority. There was clear agreement around the areas chosen, which were then supplemented with comments from the groups about why they had been chosen and relevant extracts of data, so that the “thematic” scope of each priority was clear.

Although alternative, more formal methods of consensus development could have been used (e.g., nominal group technique or the Delphi method), at the time these were felt to be too cumbersome, given the limited scope of the task at hand, and unnecessary, given that we were facilitating a large number of unmixed groups, which would lessen power issues and the risk of domination by one particular perspective (see Black, 2007). We also felt a qualitative approach to consensus development better suited the design and underpinnings of our research. We could, however, have considered recording each group’s discussions and using these transcripts to support our thematic analysis, so as to explore whether consensus was being reached with equal input from group members, or whether certain views were being suppressed. This may have proved helpful if consensus across groups had been less apparent.

Five key areas were clearly identifiable as priorities for service improvement: (a) pathways in and out of hospital, (b) providing staff with a rewarding and well-supported role, (c) communicating with families and service-users, (d) recovery-focused practice, and (e) creating a positive environment for everyone. From the remaining priorities discussed, we selected a sixth, namely, (f) recognizing and sharing good practice across professions and services, which appeared to underpin much of the feedback that we received during this phase: there were lots of examples of good practice reported to us, but these innovations were often isolated within specific teams or wards.

The research findings and the agreed priorities fit well with issues identified in the inpatient care literature (e.g., non-therapeutic ward environments, a fire-fighting approach, lack of connections with community services; Lelliot et al., 2006; The Royal College of Psychiatrists, 2011). Acute services are under particular pressure in the United Kingdom; a shortage of beds seems to be triggering a rise in involuntary admissions (Keown, Weich, Bhui, & Scott, 2011) and over-occupancy is affecting quality and safety (The Royal College of Psychiatrists, 2011). The consensus around improving recovery-focused care also resonates with evidence that the current amount and quality of therapeutic interaction available is likely to be reducing service-users’ well-being and opportunities for recovery. Radcliffe and Smith (2007) found that only 6% of inpatient service-users’ time was spent in therapeutic interaction with staff or activity groups, and Whittington and McLaughlin (2000) found that only 7% of nurses’ time was spent in therapeutic interactions. The availability of therapeutic interaction is affected by a number of factors (Nolan, Bradley, & Brimblecombe, 2011), including the loss of skilled staff to better paid jobs in community care, a resulting reliance on bank staff (meaning a reduction in continuity of care), a lack of (psychologically-oriented) staff skills (Samarasekera, 2007), a lack of time to build the relationships necessary for personalized treatment and complex interventions (Yawar, 2008), and a lack of clarity regarding the aims of hospitalization and what constitutes appropriate treatment (Nolan et al., 2011). Our participants, including staff, agreed that hospitalization must better address service-users’ needs and adequately prepare them for discharge and recovery to reduce the likelihood of relapse or long-term illness.

All feedback groups reached consensus fairly easily and recognized the typical difficulties faced by young service-users being hospitalized. However, we noticed that the staff groups had a tendency to focus more on the positive aspects of care and recent developments or improvements, whereas service-users and carers were more openly dissatisfied. We were conscious that it may prove challenging for some staff to hear just how strongly the service-users and carers felt about the problems on the wards.

Co-Design Event

The co-design event took place at a community conference center. We prepared a large team of facilitators and organizers for the event, drawing on support from postgraduate students at the university and colleagues from the NHS. This event is the critical point in EBCD—it involves enacting the shared consensus for service improvement (derived from staff, service-users and carers’ experiences) and collaborating on plans for change. It is the first time all the stakeholders are brought together (Bate & Robert, 2007). Our event involved 50 service-users, family members, inpatient and community mental health staff, and managers as collaborative partners to develop plans to address the six prioritized areas for improvement.

In EBCD, one of the key aspects of the co-design event is the sharing of filmed interview footage. Unlike traditional EBCD, we did not construct our films from the original interviews (which were audio recorded). Instead, we asked for service-user and carer volunteers from the feedback groups. Rather than using films to set the agenda, which had already been agreed at the feedback groups, we chose to use them strategically, to remind participants about the key priority areas they had identified, to set the tone of the event, and to prime the participants to work together respectfully. Specifically, we wanted to capture the “real life” positives and negatives that had been highlighted and give those staff who had not had already heard the findings the chance to get a better understanding of service-user and carers’ views.

Three EIS-users and two family members consented to taking part in video interviews, which were conducted with help from NHS staff from the “patient experience” and clinical research teams. These interviews focused on the volunteers’ experiences of inpatient care but were less open-ended and more focused than the original research interviews. The aim was to gather filmed material relating to the priority areas. Each interview lasted approximately half an hour, and each was edited down to a 2-minute clip in the form of a condensed narrative.

The volunteers were as honest and frank as participants had been in the research interviews, but we were aware that some of what they said to the interviewers might not feel safe to share in front of a large audience. We took care to edit out particularly personal details or anything that seemed unnecessarily exposing. We chose extracts that did not feel too controversial, but which linked to the priority areas. For example, one clip showed a carer talking about how she had been frantic with worry when her son went missing. She described driving around the city to look for him. When she did not find him, she had begun telephoning hospitals, eventually locating him in the local inpatient mental health facility. We linked this to our priority area of “communication” because it was clear from her story that a call from staff, saying that her son was safe, would have prevented a lot of anxiety. In another example, a service-user talked about her confusion regarding what needed to improve before she could be discharged, and we linked this to “recovery-focused practice.” Before we showed the clips, we shared them with the volunteers who were present at the event to make sure that they were still happy for them to be shown. To help the audience, we also added onscreen titles to link the narratives to the priority areas.

Great care was taken to prepare service-users, families, and staff very carefully for what to expect from the event, and we ensured that support was available for anyone who needed it. We were conscious of the current context of inpatient care in the United Kingdom (e.g., Lelliot et al., 2006), and that relationships between staff, service-users, and carers may be strained. Although most staff hold positive views toward service-users, there is a range of evidence reporting that some mental health professionals hold negative attitudes toward service-users (see Wahl & Aroesty-Cohen, 2010) and display the same stereotypes as the general public (Nordt, Rossler, & Lauber, 2006). We were aware that working in inpatient care can be very stressful (e.g., Cleary, 2004; Hummelvoll & Severinsson, 2001; Ward, 2011), that there is widespread demoralization, and that many staff are pressured by issues such as high bed occupancy, lack of training, and lack of experienced leadership (Garcia et al., 2005).

The co-design event is an opportunity to bring these potentially mistrusting groups together, and could be seen as an example of the contact hypothesis (Allport, 1954) in practice. The hypothesis, which has been robustly supported (Pettigrew & Tropp, 2006), states that intergroup contact will reduce hostility or prejudice. Four conditions are necessary: common goals, intergroup cooperation, support from authorities, and the equal status of parties (Pettigrew & Tropp, 2005). Well-meaning attempts to bring groups together may actually cause more harm than good if these conditions are not met (see Hewstone & Swart, 2011). In EBCD, the feedback groups ensure that a consensus is reached so that there are common goals. The principle of co-design, implying a partnership, encourages cooperation between groups, and senior support from management authorities should be sought from the beginning. There are, however, inequities between service-users and health care providers: service-users may be vulnerable due to stigma and distress; by contrast, individual staff members may draw authority from organizational power structures. However, the contact hypothesis literature suggests that, even when there is initially inequality between groups, creating equal status during the contact (the co-design event) is enough to promote positive intergroup attitudes (Schofield & Eurich-Fulcer, 2001; see Pettigrew & Tropp, 2005).

Researchers can encourage equality between service-users and staff by ensuring group work is undertaken with transparency (making sure everyone is clear what each others’ motivations are for taking part, that neutral and lay language is used so as not to exclude, and acknowledging tensions as they arise); empowering service-users (giving time and support to legitimize their views); acknowledging the range of identities, statuses, and discrimination experiences within the group; and managing expectations (e.g., about the limits and possibilities for change; Redwood, 2010). In our co-design event, participants were divided into mixed working groups to address one priority area per group. Each group had a “champion” (a senior member of NHS staff who could help the group to think about how ideas could be developed into plans that could feasibly be implemented—to show practical ways forward) and a “facilitator” (researchers and NHS staff with the role of supporting the group, keeping them on track, ensuring that the group worked respectfully together, empowering the service-users and carers, and acknowledging any tensions). Each group was provided with a pack of materials summarizing the remit of their group, and giving them a structured grid on which to record their ideas.

As expected, viewing the films had a powerful effect (Robert, 2013), and it was notable that the atmosphere in the room changed quickly from post-lunch high-spirits to a more sober and quiet mood. The groups then worked purposively together for an hour, to generate action plans to address their identified priority. They then had a further period to consolidate and record their plans. Groups suggested improvements as wide-ranging as a flowchart poster to help staff communicate with service-users and families (about where inpatient care falls in the typical “journey” through services), protected time to allow inpatient staff to do more therapeutic work with service-users, and additional training for staff supervision (see the appendix). Although we had anticipated that there may be some anxiety or defensiveness about being confronted with others’ perspectives of hospitalization, on the day, the event was very successful. All stakeholders were respectful of one another and worked collaboratively; there seemed to be a lot of enthusiasm and energy for the process.

Most working groups were able to generate a number of ideas to address their areas, but two groups (those co-designing improvements in “communicating with families and service-users” and “recovery-focused practice”) struggled with their remits. These two priorities could be viewed as some of the most fundamental issues facing inpatient services. Later, we wondered whether these issues were just too overwhelming or entrenched to address, or whether the researchers should have stepped into the discussions and directed them more actively. However, once we reviewed all the co-design plans together, we found that other working groups had discussed a number of overlapping issues, and had developed relevant action plans that could be implemented to address shortcomings in communication and recovery-focused practice. Overall, the feedback from the event was excellent and lots of participants volunteered to join the steering group and take the action plans forward.

Implementation

After the co-design event, action plans were handed over to a steering group, which included NHS staff, service-users, and family members. The steering group was tasked with monitoring and supporting the implementation of the action plans over the next 12 months, although we continued to attend steering group meetings and support the implementation process. Attendance at the steering group was consistent among a small group of committed staff, but there was frustration at the lack of time and organizational support available to follow through the plans. An audit of progress at 9 months showed that many plans were still at baseline or with minimal progress made. A project extension was negotiated, and the re-audit that had been due after 12 months was rescheduled for 18 months post the co-design event. A number of action plans have since been executed, but others remain unimplemented at the time of writing this article.

General Challenges

We faced a number of challenges that are generic to EBCD, and primarily these relate to driving the project, and the amount and quality of improvements achieved. From the outset, we elicited support from senior NHS staff, and a number of individuals were very committed and supportive of the project. However, the organization was being restructured, and a number of our collaborators and nominated “champions” either left or changed role, and could no longer work with us. During the research and co-design phases, this was less problematic because we were taking the lead and driving the project forward. However, once we had handed over to the steering committee, the lack of continuity in high-level support meant the responsibility for making improvements fell on a small group of individuals who were increasingly under pressure and had little “power” when it came to implementing change. An EBCD project relies on a great deal of good will, commitment, and trust between all parties, and having people both internal and external to the service in our team certainly helped project management. Donetto et al. (2014) suggest that involving service-users, carers, and staff from the beginning, in design, data collection, analysis, and development, is one way to build trust with participants, which may be especially important in sensitive contexts such as mental health. We would agree that mechanisms to encourage ownership of the project, as well as strong high-level support, are essential to guarantee the implementation and sustainability of improvements. When our 12-month report was presented to the organization’s leadership team, this was recognized, and a board member was allocated to the project’s steering group.

Despite these difficulties, it was still disappointing that more of the improvements were not implemented by the first audit. On the positive side, the service had identified some improvements that were relatively easy to implement (e.g., ensuring that when carers came to visit, they were greeted appropriately at the door of the unit; making sure that correct signage and soft furnishings were in place), and these improvements were well underway at the first audit. Less progress was being made on improvements that required input from other departments or that involved strategic, budgetary, or staffing commitments. The service had employed a number of staff in roles that suggested an organizational appetite for service-user-led collaborative work (i.e., job titles with phrases like “patient experience,” or “service quality” in them), and many of these people were involved in the project and made positive contributions to it, but the wider structure and culture of the organization did not seem to be able to respond sufficiently flexibly or directly to what is effectively “bottom-up” service development. Many of the mechanisms that make a collaborative project possible were simply not available, so that basic elements of user-led projects (e.g., accessing a budget for service-users’ and carers’ expenses and honoraria) were a continual challenge.

Neither of these challenges (governance or implementing improvements) is unique to our project. Evaluations of existing EBCD projects report that governance suffered because of high staff turnover and/or a lack of support from senior management (The King’s Fund, 2011; Piper et al., 2012; Piper et al., 2010), and that ongoing staff participation was hampered by EBCD being seen as an additional burden in an already pressurized schedule (Piper et al., 2012), with some staff feeling unable to leave their clinical duties to participate (Bowen et al., 2010; Bowen et al., 2013). Doubts regarding what service-level improvements were made (Bowen et al., 2010; Bowen et al., 2013), and concerns about the sustainability of improvements (Piper et al., 2010), also resonate with our project. In one evaluation, participants felt that EBCD was being used to justify top-down changes, rather than genuinely promoting new ideas and improvements (Piper et al., 2010), and in another, participants thought that the benefits of collaborative working were felt most by those staff groups who were already working well together (The King’s Fund, 2011). Perhaps most importantly, though, when co-designed improvements were not implemented, for whatever reasons, participants were left feeling disappointed and dissatisfied (Bowen et al., 2013; The King’s Fund, 2011), suggesting there is potential for harm, if EBCD projects are not well supported. We also felt disappointed about the lack of tangible improvements. If service-users, staff, and carers are asked to take part in service improvement, it is crucial that the changes they co-design are then implemented to avoid causing further dissatisfaction and alienation.

Concerns Specific to the Mental Health Context

The use of EBCD as a service-development approach is not subject to ethical examination in the same way that traditional research projects are. As our project began with three traditional qualitative studies, our procedures were subject to rigorous ethical review. Coming from a research perspective, we were acutely aware of the need to protect our participants from possible distress during data collection, and thus interviewers followed typical guidance for conducting qualitative interviews (e.g., King & Horrocks, 2010) and were sensitive to the difficulties some participants may have faced when recounting their (potentially distressing or traumatic) accounts. Safety procedures were included, such as contacting participants’ care teams if they were distressed by their participation in interviews. These sensitivities extended to the co-design event, where we made available a quiet room where participants could take time out, or seek support. At the event, only one person, a staff member, made use of this facility, illustrating that it is important to remember that any participant in EBCD processes may require support.

We also carefully considered the use of film (a key part of traditional EBCD) in the co-design phase. In Donetto et al.’s (2014) review of EBCD projects, they note that films are exceptionally powerful, particularly in the case of staff watching films of service-users’ experiences. Films seem to act as a humanizing mechanism for staff, allowing them to reconnect with service-users on an emotional level. Far fewer of the projects in Donetto et al.’s (2014) report also filmed staff, and some staff were reportedly reluctant to be filmed. It may be that audio recording staff interviews was more palatable, and it certainly enabled us to collect rich staff accounts. Our choice not to include films of staff experience at the event was partly to address the inequities in the group. We felt that service-users and carers were potentially disadvantaged, because there were a large number of nursing staff and managers at the event, whose experiences, to a larger extent, fall into line with the dominant structures and discourse of the NHS. Moreover, we prioritized service-users’ voices because the core aim of an EBCD project is to improve services for that group.

Film is additionally challenging in the context of mental health, because anonymity is likely to be much more important than in physical health settings, due to the high levels of associated stigma. Filming of service-users clearly offers no protection from identification, and film preserves that their testimony, their state of health, how they choose to identify themselves (e.g., as a service-user), and the experiences they have had, indefinitely. The legacy of films needs to be carefully considered, and films should be protected from being used for alternative agendas. Identifying service-users through showing films could potentially put their care at risk (or be perceived as doing so, thus affecting the trust between service-users and the service). Similarly, filming staff members may expose them to criticism or cause repercussions for their career progression, or be perceived to have done so. The films that we did create and show were treated as any other qualitative data, and kept securely by the researchers. An alternative would have been to give the films to the volunteers to keep for themselves.

Choosing what material to share at the co-design event, even in the form of anonymous quotations from the original interviews, was also a matter for careful consideration. While not wanting to “censor” personal accounts, we weighed this against the potential impact on the audience (so, we did not include details of serious incidents that had occurred on wards). We knew the data were powerful, but we believed they could also be overwhelming, distressing, and gratuitous in some circumstances. Where the same point could be made less graphically, we chose not to include certain material. We had to be sure that the experiences were being shared for the “right” reasons, and not to dramatize or aggrandize experiences, while also not shying away from difficult and challenging aspects.

Successes

Having a formal qualitative research phase at the start of our EBCD project allowed us to develop the co-design phase from a solid evidence base. We were confident that we had a plausible and coherent analysis, based on in-depth interviews, and that there were a number of shared concerns from staff, families, and service-users. Although Robert (2013) suggests semi-structured interviews, non-participant observation, and a thematic analysis of interview transcripts, existing EBCD projects in physical health have used a variety of methods to collect experience data (including qualitative questionnaires, focus groups, workshops, observation, diary methods, video-booths, and visual methods; Donetto et al., 2014), and it is unclear what methods of analysis, if any, have been used.

Including feedback groups before the main co-design event, as Robert (2013) advises, allowed us to “take the temperature” of the different stakeholders’ responses to the research material, and to better understand the likely challenges ahead at the co-design event. What we learnt enabled us to run the co-design event safely and productively. The event was the highlight of the project, and the formal feedback was excellent, with participants commenting that it offered the opportunity to begin positive work toward change, and was “rejuvenating,” “valuable,” and “productive.” One carer commented on the bravery of the people who had shared their experience, and a staff member noted how powerful the films were and how important it was for staff to recognize that the service they think they provide is not always what service-users experience.

At the first audit, 9 months after the co-design event, approximately a third of the improvements had been implemented and there was demonstrable progress on another third. Given the systemic difficulties we encountered, the improvements seen in this project make it at least partially successful. There is an ongoing commitment from the steering group, who continue to press forward with implementation, despite the challenges, and we are hopeful that there will be further progress by the re-audit. We should not underestimate what a challenge it is to improve psychiatric inpatient care, given its complexity and the number of issues currently encountered in many services. Our approach to EBCD involved carefully reflecting on each stage before moving forward, meaning that the project lasted longer than many others, but was rigorous and ethically sound. It offers some evidence that it is feasible to use EBCD in mental health settings, provided there is consideration of the ethical issues at hand, and a sensitive and careful approach is taken.

The Implications of Using EBCD

Running an EBCD project is dependent on key “brokers” who can keep the project on track and implement the improvements (Robert, 2013). We believe that allied health professionals (AHP), such as psychologists and therapists, may be in a good position to take on a leadership role (with senior managerial support) for this kind of project, because they are often less directly implicated in the more coercive aspects of psychiatric care, and thus may be better placed to mediate between staff, service-users, and carers. Certainly, experience and skills in facilitating group work are an advantage. External research staff may not face the same issues gaining the trust of service-users and families, but their lack of local knowledge is likely to be detrimental in setting up, managing, and implementing the project. If external researchers are attempting to run EBCD projects in mental health settings, we would advise either a close relationship with an internal partner (as we had) or for the researcher to be embedded, at least temporarily, within the service-environment.

An EBCD project is only successful if improvements are made to the service, but it is this that requires the greatest support from outside the core project team. A service or organization may make a commitment to listen to service-users, carers, and staff through an EBCD project, but if the organizational will, support, and mechanisms are not readily available, the project will not fulfill its potential. Senior organizational support is required from the outset. A number of different supporters is also advisable, especially in times of organizational uncertainty, when key staff can suddenly move into other posts with new remits.

If neither high-level commitment nor structural mechanisms to support user-led projects are available, then keeping the scope of the EBCD project narrow and focused may be one way to ensure successful projects. If change can be managed by a committed team, at a local level, then implementing improvements without considerable high-level support may be possible. Yet, even with support and the organizational structures in place to implement change, it is likely that projects will need additional guidance to make improvements in complex services, such as mental health settings. Currently, the EBCD toolkit (The King’s Fund, 2012) does not discuss the implementation phase. Perhaps more structured and detailed guidance on how to facilitate change in these complex environments once the co-design event is over is required.

Questions for Future Research and Method Development

EBCD offers the potential to reorganize services on the basis of experience, but this is not the only form of evidence that should be taken into account when designing health care. It is important for future research to consider if and how EBCD can be dovetailed with evidence-based practice, including statutory guidance (e.g., NICE, 2011b), and practice-based evidence. As EBCD is taken up more widely, it is possible that conflicts may arise between the formal, informal, and experiential evidence about what constitutes best care. Such conflicts will need to be negotiated carefully.

Although EBCD is designed to support local change, the findings of any EBCD project may also apply to other settings. Our own findings about the difficulties of hospitalization in early psychosis are likely to be transferable beyond the service we worked with. If findings can be generalized, it is possible that co-designed improvements may also have reach beyond the local context. Whether and how locally designed interventions, toolkits or templates, may be implemented on a wider scale needs investigation.

EBCD seems to have the potential for meeting the complex needs of service design and quality in times of austerity. The changes that need to take place in mental health, particularly in inpatient care, are much more likely to be successful if all stakeholders are committed to a shared narrative (Gilburt & Peck, 2014), and EBCD seems to offer a way to help different parties better understand each others’ perspectives. Yet, we must also consider whether we are simply asking too much of EBCD participants in expecting them to generate solutions to very difficult problems in a short space of time. Perhaps supporting co-design participants, for example, by providing research to stimulate ideas, providing opportunities to develop solutions in a longer workshop format, or providing training, may help participants tackle the difficult job of co-designing improvements. We need to understand whether feeding additional evidence or skills into the EBCD process is useful, or whether it could stymie innovation, or worse reinforce the perception that the researchers and clinicians are the “experts” and that service-user involvement is mere lip-service.