Supporting Recovery by Improving Patient Engagement in a Forensic Mental Health Hospital

Abstract

BACKGROUND: Mental health services are shifting toward approaches that promote patients’ choices and acknowledge the value of their lived experiences. OBJECTIVE: To support patients’ recovery and improve their experiences of care in a Canadian forensic mental health hospital, an intervention was launched to increase patient engagement by establishing a peer support program, strengthening a patient advisory committee, and creating a patient-led research team. DESIGN: The effect of the intervention on patient- and system-level outcomes was studied using a naturalistic, prospective, longitudinal approach. Quantitative and qualitative data were gathered from inpatients and service providers twice during the 19-month intervention. RESULTS: Despite succeeding in supporting patients’ participation, the intervention had minimal impacts on internalized stigma, personal recovery, personal empowerment, service engagement, therapeutic milieu, and the recovery orientation of services. Peer support demonstrated positive effects on internalized stigma and personal recovery. CONCLUSIONS: Strengthening patient engagement contributes toward improving experiences of care in a forensic hospital, but it may have limited effects on outcomes.

Keywords

forensic mental health patient engagement recovery peer support participatory action research

The Concept of Patient Engagement

Constructs such as patient engagement, recovery-oriented care, patient-centered care, consumer involvement, and collaborative care are often used synonymously to describe processes that, among other things, provide patients with self-determination and control over health care decisions. Although their emphasis and underpinning philosophies vary (Forbat, Hubbard, & Kearney, 2009), one aspiration shared by these concepts is to move away from paternalistic health care practices, toward systems of care that support patients’ choices and acknowledge the value of their lived experiences (Forbat et al., 2009; Liberman & Kopelowicz, 2005; Tambuyzer, Pieters, & Van Audenhove, 2011). For instance, recovery-oriented service systems support the autonomy and self-determination of people with mental illness so that they can reclaim their rights as citizens and regain meaning in their lives (Le Boutillier et al., 2011). Similarly, patient-centered health care emphasizes power sharing and promotes relationships between patients and their health care providers that are nonhierarchical and respectful (Mead & Bower, 2000).

Patient engagement has been conceptualized by Hanley et al. (2004) on a continuum representing a diverse array of strategies and approaches. At one end of the continuum, patients are passive service recipients who have no involvement in health care decisions, whereas at the other end, patients are in full control of health care processes and possess the authority to make health care decisions. Somewhere between these two extremes, patients are consulted or engaged in collaborations, whereby they have the opportunity to influence health care decisions and power is shared with service providers, health care managers, and policy makers. Building on this continuum, Forbat et al. (2009) articulate a model outlining four distinct domains of patient engagement activities associated with varying ideological drivers and levels of participation. In the first set of activities, patients are consumers who engage in health care by purchasing and choosing the services they receive. In the second, patients are citizens who, according to social–democratic principles, engage by influencing the development of health care policy and services. In the third set of patient engagement activities, patients are partners in health care and use their experiential knowledge to inform health care practices in relation to their own care. Last, patients are researchers and engage in health care by cocreating and mobilizing knowledge through the use of community-based participatory research approaches. These activities occur at different organizational levels of care, including at the micro-level, where individual patients and service providers work together in therapeutic relationships, all the way through to the macro-level, where patients are involved in developing health care policy (Tambuyzer et al., 2011).

As is evident from the conceptual literature, patient engagement is a complex, multifaceted construct involving different approaches, multiple levels, and numerous processes. Here, we use patient engagement to refer to the active participation and meaningful involvement of patients in a range of activities and decision-making processes in the health care system (Tambuyzer et al., 2011). The term patient is used herein to describe an individual who is receiving inpatient mental health care. Although this concept carries certain symbolic meaning that may be perceived as being incongruent with the recovery paradigm, we believe it is appropriate for describing a forensic mental health hospital population. The terms service-user and consumer imply choice in mental health services, which should be distinguished from the involuntary nature of forensic mental health services. Moreover, patient is the preferred term for many people receiving professional mental health services (Simmons, Hawley, Gale, & Sivakumaran, 2010), and was endorsed by our study participants.

Evidence of Effectiveness

It is important to consider the goals of patient engagement before embarking on a discussion of its effectiveness. In an excellent conceptual review article, Tambuyzer et al. (2011) identify four main justifications for patient engagement in health care systems. First, it is the right thing to do—giving patients choice and involvement in health care decisions is in keeping with ethical and democratic principles (the principle argument). Second, it enriches health—patient engagement is believed to improve therapeutic outcomes (the therapeutic argument). Third, it enhances services—patient engagement is thought to improve the quality of care, such as making services more accessible and responsive to patients’ needs (the quality argument). Fourth, it helps organizations meet standards—patient engagement increases the legitimacy and credibility of a health care organization especially if they are required to comply with government regulations and accreditation standards mandating patient involvement (the political argument).

We would add two other justifications based on our own observations and experience. Patient engagement is often framed as a way to reduce costs (the efficiency argument). For instance, focusing on disease self-management shifts the responsibility to the patient and, consequently, takes some of the burden away from paid professionals. These patient-centered interventions under engagement paradigms, such as peer support, cost significantly less than traditional health care services (Pistrang, Barker, & Humphreys, 2008). Finally, organizations that prioritize engagement with patients are more attractive for consumers, which is especially important in competitive markets (the commercial argument).

Although there are many perceived benefits associated with patient engagement, numerous literature reviews have underscored that there is a dearth of empirical findings about its effects (positive or negative) for patients, service systems, and broader populations (Cayton, 2004; Crawford et al., 2002; Gregory, 2007; McEvoy, Keenaghan, & Murray, 2008; Ridley & Jones, 2002; Simces, 2003). A recent systematic review of research about consumer involvement in health care concluded that there is a “paucity of evidence for most outcomes, including participation or response rates, decisions, health care outcomes, satisfaction, impacts on participating consumers, and cost” (Nilsen, Myrhaug, Johansen, Oliver, & Oxman, 2010, p. 19). However, the review did find reasonable evidence to suggest that the quality (i.e., relevance and readability) of patient information materials is likely to be improved by involving consumers in their development (Nilsen et al., 2010). Other reviews have found that involving consumer-providers on assertive community treatment and case management teams can enhance treatment engagement and therapeutic relationships (Wright-Berryman, McGuire, & Salyers, 2011). Given the lack of data about its effectiveness, it may be concluded that recent efforts to increase patient engagement in health care, at the present moment, are largely built on an ideological foundation with the hope that such a paradigm shift will produce improvements and efficiencies in the health care system.

Patient Engagement in Forensic Settings

On the face of it, patient engagement seems contrary to the sometimes involuntary, coercive, and paternalistic practices of the traditional forensic mental health system. Forensic mental health describes a specialized area that involves the assessment and treatment of people with mental disorders that have engaged in, or are at risk for engaging in, criminal behavior (Mullen, 2000). A growing number of authors have wrestled with the theoretical and pragmatic issues associated with incorporating patient engagement practices and recovery principles into forensic mental health settings (Corlett & Miles, 2010; Geller, 2012; Green, Batson, & Gudjonsson, 2011; Gudjonsson, Webster, & Green, 2010; Hillbrand, Young, & Griffith, 2010; McLoughlin, Geller, & Tolan, 2011; Mezey, Kavuma, Turton, Demetriou, & Wright, 2010; Simpson & Penney, 2011). The collective wisdom from this body of literature is that patients can be engaged in forensic services in ways that are consistent with recovery-oriented, patient-centered care approaches.

Our own research with an involuntarily detained patient population has demonstrated that the characteristics of patient-centered, recovery-oriented care approaches are, in fact, present in a secure forensic hospital (Livingston, Nijdam-Jones, & Brink, 2012). However, safety issues in forensic settings necessitate the use of special methods when implementing patient engagement approaches. For instance, patients may be given opportunities to make decisions about some aspects of their care but not others. The notion of patients as being autonomous agents in decision making, who are equal partners in their health care, is also challenged by a subgroup of people with severe, persistent mental illness who may not have the adequate insight or cognitive capacity to make appropriate decisions pertaining to their personal health and safety. In such circumstances, coercive interventions such as involuntary hospitalization may be a necessary stepping stone for individuals to reactivate the recovery process and to regain the cognitive capacity to collaborate in their own care (Geller, 2012). Despite having their liberty and freedom curtailed, opportunities exist to meaningfully involve forensic inpatients in the design, delivery, and evaluation of services.

We previously published cross-sectional data regarding the perceptions of recovery-oriented, patient-centered care among patients and service providers in a forensic hospital (Livingston et al., 2012). Here, we build on these data by presenting the results of an evaluation study examining the effectiveness of a patient engagement intervention in a forensic hospital.

Method

Study Design

A naturalistic, prospective, longitudinal approach was employed to assess the degree to which a 19-month patient engagement intervention influenced patient- and system-level outcomes in a Canadian forensic hospital. A repeated-measures design was employed for the patient participants who completed an interview at both baseline (T1) and follow-up (T2). In contrast, a successive independent samples design was used with the two groups of service providers who completed a survey either at T1 (n = 28) or T2 (n = 22). The average daily census in the 190-bed forensic hospital was 172 adult patients during the intervention timeframe, of which approximately 85% were men and 75% were adjudicated “Not Criminally Responsible on Account of Mental Disorder” (NCRMD); Canada’s insanity defense). The research protocol was approved by the research ethics boards of the University of British Columbia and the British Columbia Forensic Psychiatric Services Commission. Informed consent was obtained from each participant.

Intervention

The patient engagement intervention was composed of three components that sought to support patients’ recovery and improve their experiences of care by strengthening patient engagement in a forensic hospital. The goals of the intervention were to (a) implement a range of patient engagement strategies in a forensic hospital, (b) assess whether the strategies improved patients’ experiences of care, and (c) evaluate whether such strategies produced positive outcomes for patients and the broader service system. The specific components included a peer support program, a patient advisory committee, and a patient research team. The patient engagement intervention was launched in early 2011 and evaluated across 19 months. The entire initiative cost approximately $200,000 CDN. The intervention was targeted principally toward patients who were adjudicated NCRMD, involuntarily detained in hospital, and receiving treatment services. As this was a naturalistic study, patients freely decided whether or not to take part in the patient engagement intervention and they were not ordered or allocated into any of its components.

Peer support program

A peer support worker—someone in recovery with mental health and substance use disorders with training in counseling—was hired and supported to design, implement, and deliver a peer support program (program manual available by contacting the principal author). The program included weekly mutual support groups (each 1 hour in length) and individual-based peer support (5-30 minutes in length), which enabled patients to connect with a peer support worker on a one-on-one basis. It was designed to create safe (i.e., confidential) spaces for patients to openly discuss recovery issues regarding their mental illness and substance use problems with others who could relate to their experiences and who could offer support. On a day-to-day basis, the peer support worker acted as a role model for recovery and shared information about their own recovery to engage patients in a reciprocal relationship. Within the evaluation timeframe, 127 support group meetings for men and women were provided, with an average attendance of 6.28 patients (SD = 1.73, MIN = 4, MAX = 12) in the men’s group and 3.42 patients (SD = 1.33, MIN = 1, MAX = 8) in the women’s group. There were 101 patients who received individual peer support, with an average of 6.31 individual sessions (SD = 7.07, MIN = 1, MAX = 46) provided to each patient.

Patient Advisory Committee

Support was provided to revitalize a preexisting Patient Advisory Committee (PAC). The PAC was composed of patient representatives and staff from the forensic hospital who gathered monthly to discuss hospital-wide issues and concerns. Although the PAC had been in operation for several years prior to the patient engagement intervention, interest from patients was waning, meetings were held infrequently, and connections with senior management did not exist. The intervention produced the following improvements to the PAC: (a) ensured that management representatives consistently attended monthly PAC meetings to listen and attend to issues raised by patients and to elicit patient feedback on organizational initiatives; (b) supported existing staff who were passionate about patient engagement to provide additional assistance to the PAC; (c) developed and ensured strong and diverse patient representation on the PAC; (d) established regular meetings between the PAC Chair (patient) and the Vice President of Medical Affairs; (e) established positions for the PAC Chair on operational committees in the forensic hospital; and (f) strengthened communication between the PAC and the nine individual units in the forensic hospital, especially with the five maximum secure units housing 98 patients who were unable to attend PAC meetings because of security reasons. During the evaluation period, 18 PAC meetings were held with an average attendance of 7.78 patients (SD = 1.31, MIN = 5, MAX = 11).

The PAC has grown to be an important venue for patient involvement in decision making at the forensic hospital, which is evidenced by the growth in patient-initiated improvements that were achieved following the implementation of the patient engagement intervention. For instance, the PAC was instrumental in reinstating the availability of caffeinated coffee in the mornings for patients at the forensic hospital, which had been prohibited for several years on account of staff concerns about caffeine abuse and extending the evening “lights out” curfew on weekends and holidays.

Peer research team (Team PEER)

Using a Participatory Action Research approach, a patient-led research team was created. The team was composed of peer researchers who lived with mental illness, were adjudicated NCRMD, and were involuntarily detained in the forensic hospital. The process began with eight patients, of which six remained involved by the end of the study. One person dropped out early in the process on account of lack of interest in the topic ultimately chosen by the research group, and another person discontinued late in the process to concentrate more fully on other commitments, such as work, as they prepared for discharge. Together with academic researchers, the peer researchers (self-named Team PEER: Patients Empowered and Engaged as Researchers) planned, designed, and carried out a study on treatment planning—a topic they deemed to be critical to their experiences of care at the forensic hospital (Livingston, Nijdam-Jones, & Team P.E.E.R., 2013). There were 55 Team PEER meetings held during the evaluation period, with an average attendance of 6.10 patients (SD = 0.99, MIN = 3, MAX = 8).

Study Participants

Patients were eligible to participate in the evaluation study if they were (a) at least 19 years of age, (b) English-speaking, (c) receiving treatment services for at least 1 month at the forensic hospital, and (d) deemed by their psychiatrist as capable for study participation (i.e., had sufficient cognitive capacity to provide informed research consent, not a current risk to staff). Participation in the intervention was not required for participation in the study. Approximately 30% of all eligible patients at the forensic hospital participated in this study: 30 patients enrolled at baseline (T1), of which 83.3% (n = 25) were reinterviewed at follow-up (T2). Only data for the 25 patients who had completed T1 and T2 interviews were included in the analyses. Most of the patients were men (80.0%, n = 20), White (88.0%, n = 22), diagnosed with schizophrenia or other psychotic disorders (88.0%, n = 22), and had a history of problematic substance use (52.0%, n = 13). The average age was 42 years (SD = 10.78, range = 23-63). The median length of stay at the forensic hospital was 23 months (M = 49.88, SD = 58.06, range = 3 months to 17 years). A large majority of the patients (84.0%, n = 21) had participated in at least one component of the patient engagement intervention. More than three quarters (76.0%, n = 19) of the patients were engaged in the peer support program and had attended a median of five individual- or group-based sessions (M = 13.16, SD = 17.12, range = 1-49) with the peer support worker. One third (36.0%, n = 9) were involved in the PAC and had attended an average of 8.67 group meetings (SD = 4.97, range = 1-13). Almost one third (32.0%, n = 8) were involved in Team PEER and had attended an average of 35.03 group meetings (SD = 13.37, range = 5-47).

Service providers were eligible to participate in the evaluation study if they had worked in a clinical or therapeutic role at the forensic hospital for at least 6 months. Twenty-eight service providers participated at baseline (T1) and 22 participated at 1-year follow-up (T2). More than half of the service providers were women (T1: n = 14, T2: n = 13) and were an average of 44 years of age (T1: M = 43.4, SD = 10.8; T2: M = 43.3, SD = 10.2). A diverse range of professional groups were represented in the service provider sample, including nurses, recreational and occupational therapists, counselors, psychiatrists, and social workers. Data from T1 and T2 could only be linked for nine service provider participants, which was either on account of technical issues with linking the anonymous data or because the two groups (T1 and T2) were largely composed of different individuals. As a result, we employed a successive independent sample approach with the service provider participants by treating T1 and T2 data as though they were produced from two different samples.

Data Collection

Patient interviews

Semistructured interviews were conducted in private and lasted approximately 60 minutes. T1 interviews were administered near the beginning of the patient engagement intervention (May to July 2011) and were followed by a second interview (T2) administered an average of 8.44 months (SD = 1.53) later. A 9-month interval between T1 and T2 interviews was chosen to minimize the loss of data that occurs when patients are discharged from hospital. A few patients were discharged during the study period and received an early T2 interview. The interviews contained standardized quantitative measures (described below). At T2, patients were also asked to rate the degree to which they perceived improvements across different themes of patient engagement, namely, valuing patients’ choices, involving patients in their own care, supporting illness self-management, and providing patients with opportunities to give feedback. Additionally, patients were asked open-ended questions at T2 about the three components of the intervention (i.e., peer support, PAC, and Team PEER). Interviews were audio recorded and transcribed verbatim. Patients were paid $10 for participating in each interview.

Service provider survey

Service providers completed an anonymous paper-based or online questionnaire. The T1 survey was completed near the beginning of the patient engagement intervention (May to July 2011) and was followed by a second survey (T2) that was administered 1 year later (May to July 2012). The questionnaire contained standardized quantitative measures of recovery-oriented care and therapeutic milieu (described below). At T2, service providers also rated the amount of change they had observed that could be attributed to the patient engagement intervention. Furthermore, service providers answered open-ended questions at T2 about each intervention component and whether they perceived that patients’ experiences at the forensic hospital had improved during the study period. Survey participants were remunerated by being entered into a random draw for literature on recovery and patient-centered care.

Outcome Measures

Based on the limited literature, we hypothesized that the patient engagement intervention would produce reductions in internalized stigma, along with increases in personal recovery, personal empowerment, and service engagement among patients (Corrigan, Roe, & Tsang, 2011; Resnick & Rosenheck, 2008; Sidani, 2008; Tambuyzer et al., 2011; Tempier et al., 2010; van Gestel-Timmermans, Brouwers, van Assen, & van Nieuwenhuizen, 2012; Warner, 2010). Significant improvements were also anticipated with respect to how patients and service providers perceived the therapeutic milieu and recovery orientation of services at the forensic hospital.

Personal recovery

The personal recovery process was assessed using the Mental Health Recovery Measure (MHRM; Bullock, 2005). The MHRM contains 30 items and seven subscales that reflect different stages of recovery: overcoming stuckness, self-empowerment, learning and self-redefinition, basic functioning, overall well-being, new potentials, and advocacy/enrichment. Cronbach’s α was .88 (T1) and .90 (T2).

Service engagement

Engagement in mental health services was measured using the Singh O’Brien Level of Engagement Scale (SOLES; O’Brien, White, Fahmy, & Singh, 2009). The SOLES contains 16 items and assesses two domains: acceptance of need for treatment and perceived benefit of treatment. Cronbach’s α was .91 (T1) and .87 (T2).

Internalized stigma

Subjective, internalized experiences of stigma were assessed using the Internalized Stigma of Mental Illness scale (ISMI; Ritsher, Otilingam, & Grajales, 2003). Internalized stigma describes a process whereby persons with mental illness endorse stereotypes about their illness, consider such stereotypes to be self-relevant, and believe themselves to be devalued members of society (Livingston & Boyd, 2010). The 29-item measure covers five domains: alienation, stereotype endorsement, discrimination experiences, social withdrawal, and stigma resistance. Cronbach’s α was .92 (T1) and .91 (T2).

Empowerment

Personal empowerment was assessed using the Making Decisions Empowerment Scale (MDES; Rogers, Chamberlin, Ellison, & Crean, 1997). The 28-item instrument contains five domains: self-esteem/self-efficacy, power-powerlessness, community activism and autonomy, optimism and control over the future, and righteous anger. Cronbach’s α was .73 (T1) and .70 (T2).

Recovery-oriented care

The degree to which services were aligned with the principles of the recovery model was measured using the Recovery Self Assessment Scale (RSA; O’Connell, Tondora, Croog, Evans, & Davidson, 2005). The RSA has 32 items and five domains: life goals, service involvement, diversity of treatment options, choice, and individually tailored services. Patients completed the “person in recovery” version and service providers completed the “provider” version of the RSA. Cronbach’s α was .90 (T1) and .94 (T2) for patients and .89 (T1) and .93 (T2) for service providers.

Therapeutic milieu

Hospital milieu was assessed with the Essen Climate Evaluation Schema (EssenCES), which was designed for use in forensic inpatient settings (Schalast, Redies, Collins, Stacey, & Howells, 2008). The EssenCES is a 15-item scale with three domains: patients’ cohesion, experienced safety, and therapeutic hold. The EssenCES was administered to patients and services providers. Cronbach’s α was .78 (T1) and .83 (T2) for patients and .78 (T1) and .32 (T2) for service providers. Because of the low alpha for service providers at T2, we identified four items (4, 9, 10, and 16) that had weak correlations (r < .10) with the total average score and recalculated an adjusted total average EssenCES score (with the four items removed) for the service providers at T1 and T2. Cronbach’s α for the modified 11-item EssenCES was .78 (T1) and .68 (T2) for service providers.

Data Analysis

Quantitative analyses were performed using SPSS version 19. For the data produced by the standardized instruments, we first performed Student t tests to contrast the average total score for each outcome measure at T1 and T2 among the patients and service providers. Data for the 25 patients who completed interviews at T1 and T2 were analyzed using a paired samples t test. Data for the service providers who completed a survey either at T1 (n = 28) or T2 (n = 22) were analyzed using an independent samples t test.

Using patient attendance data for each of the intervention components, we trisected the patients into three groups to represent their level of participation in the patient engagement intervention, with a third of the participants in low, moderate, or high categories. We performed analyses of variance to assess whether mean change (T2 minus T1) on each outcome differed between these three groups. Finally, we conducted bivariate correlation analyses using the Pearson product–moment correlation coefficient to examine whether or not participation in the peer support program, PAC, or Team PEER (Yes = 1, No = 0) was associated with the mean change from T1 to T2 on each outcome measure.

Descriptive statistics were conducted with the quantitative data produced from the four evaluation questions that were customized to assess the amount and nature of change perceived by the patients and service providers. Similar to the outcome data, we trisected the patients into three groups according to their level of participation in the intervention (low, moderate, and high) and performed analyses of variance to assess whether these three groups differed in the amount of change they had perceived. We also performed Student t tests to determine whether average scores on each of the evaluation questions differed significantly between patients who did or did not participate in the intervention components.

Qualitative analysis was performed by two of the authors using NVivo version 9. The analytic procedure was guided by qualitative description methods (Neergaard, Olesen, Andersen, & Sondergaard, 2009). First, the participants’ qualitative responses were reviewed and coded. Then, the codes were organized and sorted to identify similar phrases and ideas that were salient to the research questions. Finally, codes with similar features were collapsed into a smaller subset of broad themes.

Results

Patient-Level Outcomes

Personal recovery

Overall, patients’ average scores on the MHRM were not statistically different at T1 and T2. Change in personal recovery from T1 to T2 was statistically similar for patients with low (n = 9, M_diff = 1.11, SD_diff = 5.23), moderate (n = 7, M_diff = −0.57, SD_diff = 10.36), or high (n = 8, M_diff = 0.38, SD_diff = 8.11) levels of participation in the intervention, F(2, 23) = 0.89, p > .05. Similarly, mean change in MHRM scores was not significantly associated with taking part in peer support (r = .05, p > .05), the PAC (r = .22, p > .05), or Team PEER (r = −.02, p > .05). However, there was a significant association between attending at least one peer support group (omitting one-to-one peer support) and improvements in personal recovery from T1 to T2 (r = .40, p = .05). Whereas patients who participated in the peer support groups reported increases in personal recovery (n = 10, M_diff = 3.90, SD_diff = 8.17), those who did not participate in the peer support groups reported decreases in MHRM scores (n = 14, M_diff = −2.14, SD_diff = 6.32), t(22) = −2.05, p = .05 (see Table 1). In summary, the intervention did not have a significant effect on patients’ levels of personal recovery, as measured by the MHRM, for the subgroup of patients who participated in an interview. The one exception was that attendance in a peer support group may have had positive effects on personal recovery among the study participants.

Table 1.

Outcome Data at T1 and T2 for Patients and Service Providers.

	T1			T2
Measures	N	M	SD	N	M	SD	t Test (2-tailed)^a
Patients
Personal recovery (MHRM total)	24	86.25	13.56	24	86.63	14.00	t(23) = −0.24, p > .05
Service engagement (SOLES total)	25	6.66	1.96	25	6.91	1.88	t(24)= −1.03, p > .05
Internalized stigma (ISMI total)	23	2.07	0.42	23	2.08	0.40	t(22) = −0.21, p > .05
Personal empowerment (MDES total)	25	2.86	0.25	25	2.91	0.24	t(24) = −1.54, p > .05
Recovery-oriented care (RSA total)	24	3.11	0.66	24	3.09	0.79	t(23) = 0.16, p > .05
Therapeutic milieu (EssenCES total)	24	2.07	0.51	24	2.03	0.55	t(23) = 0.62, p > .05
Service providers
Recovery-oriented care (RSA total)	27	3.03	0.60	21	2.98	0.54	t(42.24) = 0.91, p > .05
Therapeutic milieu (EssenCES total)	28	1.50	0.58	22	1.35	0.40	t(47.31) = 1.05, p > .05

Note. MHRM = Mental Health Recovery Measure; SOLES = Singh O’Brien Level of Engagement Scale; ISMI = Internalized Stigma of Mental Illness scale; MDES = Making Decisions Empowerment Scale; RSA = Recovery Self Assessment; EssenCES = Essen Climate Evaluation Schema.

Patient participants’ data analyzed with paired sample t test (2-tailed). Service provider participants’ data analyzed with independent samples t test (2-tailed).

Service engagement

Patients’ average ratings on the SOLES were statistically similar at T1 and T2. Change in service engagement from T1 to T2 was statistically similar for patients with low (n = 10, M_diff = 0.56, SD_diff = 1.55), moderate (n = 7, M_diff = -0.34, SD_diff = 0.71), or high (n = 8, M_diff = 0.37, SD_diff = 1.03) levels of participation in the intervention, F(2, 24) = 1.21, p > .05. Similarly, mean change in SOLES scores was not significantly associated with taking part in peer support (r = .05, p > .05), the PAC (r = −.17, p > .05), or Team PEER (r = −.05, p > .05). In summary, the intervention did not have a significant effect on levels of service engagement, as measured by the SOLES, for the subgroup of patients who participated in an interview.

Internalized stigma

Patients’ average scores on the ISMI were statistically similar at T1 and T2. Change in internalized stigma from T1 to T2 was statistically similar for patients with low (n = 8, M_diff = −0.12, SD_diff = 0.25), moderate (n = 7, M_diff = 0.01, SD_diff = 0.35), or high (n = 8, M_diff = −0.09, SD_diff = 0.21) levels of participation in the intervention, F(2, 22) = 1.24, p > .05. Similarly, mean change in ISMI scores was not significantly associated with taking part in the PAC (r = .00, p > .05) or Team PEER (r = −.25, p > .05). However, there was a significant correlation between whether or not patients took part in individual- or group-based peer support and change in internalized stigma from T1 to T2 (r = −.43, p < .05). On average, patients who participated in peer support experienced slight decreases in internalized stigma (n = 18, M_diff = −0.05, SD_diff = 0.27), whereas those who did not participate in peer support reported increases in ISMI scores (n = 5, M_diff = 0.25, SD_diff = 0.16), t(21) = 2.21, p < .05. In summary, the intervention as a whole did not have a significant effect on the level of internalized stigma, as measured by the ISMI, for the subgroup patients who participated in an interview. The one exception was that peer support may have had positive effects on internalized stigma among the study participants.

Personal empowerment

Patients’ average scores on the MDES were statistically similar at T1 and T2. Change in personal empowerment from T1 to T2 was statistically similar for patients with low (n = 10, M_diff = 0.00, SD_diff = 0.18), moderate (n = 7, M_diff = 0.08, SD_diff = 0.10), or high (n = 8, M_diff = 0.08, SD_diff = 0.17) levels of participation in the intervention, F(2, 24) = 0.72, p > .05. Similarly, mean change in MDES scores was not significantly associated with taking part in peer support (r = .04, p > .05), the PAC (r = .10, p > .05), or Team PEER (r = .01, p > .05). In summary, the intervention did not have a significant effect on levels of personal empowerment, as measured by the MDES, for the subgroup of patients who participated in an interview.

System-Level Outcomes

Recovery-oriented care

Patients’ and service providers’ average ratings on the RSA did not change significantly from T1 to T2. Since the overlap between our service provider samples violated the assumption of independence, the analysis was performed again by excluding the nine service providers who completed a survey at T1 and T2, and similar findings were produced, t(26.96) = 0.13, p > .05. Change in perceptions of recovery-oriented care from T1 to T2 was statistically similar for patients with low (n = 9, M_diff = 0.08, SD_diff = 0.71), moderate (n = 7, M_diff = −0.25, SD_diff = 0.41), or high (n = 8, M_diff = 0.08, SD_diff = 0.38) levels of participation in the intervention, F(2, 23) = 0.94, p > .05. Similarly, mean change in RSA scores among patients was not significantly associated with taking part in peer support (r = .19, p > .05), the PAC (r = −.02, p > .05), or Team PEER (r = −.19, p > .05). In summary, the intervention did not have a significant effect on perceptions of recovery orientation of services, as measured by the RSA, for the subgroup of patients and service providers who participated in an interview.

Therapeutic milieu

Patients’ and service providers’ average scores on the EssenCES did not change significantly from T1 to T2. Performing the analysis without the nine service providers who completed a survey at T1 and T2 produced similar findings, t(29.81) = 0.45, p > .05. Change in perceptions of therapeutic milieu from T1 and T2 was statistically similar for patients with low (n = 9, M_diff = −0.04, SD_diff = 0.27), moderate (n = 7, M_diff = −0.10, SD_diff = 0.36), or high (n = 8, M_diff = 0.02, SD_diff = 0.31) levels of participation in the intervention, F(2, 23) = 0.30, p > .05. Similarly, mean change in EssenCES scores among patients was not significantly associated with taking part in peer support (r = .11, p > .05), the PAC (r = −.03, p > .05), or Team PEER (r = .03, p > .05). In summary, the intervention did not have a significant effect on perceptions of therapeutic milieu, as measured by the EssenCES, for the subgroup of patients and service providers who participated in an interview.

Perceived Change

Quantitative findings

As is summarized in Table 2, the majority of patients perceived that at least “some” progress had been made during the study period with respect to patient engagement at the forensic hospital. The greatest improvement was observed in relation to valuing patients’ preferences about treatment decisions, for which 45.8% (n = 11) of patients indicated that “moderate” or “extreme” improvements had been made in the past year. The least improvement was perceived in the areas of educating and supporting patients to self-manage their illness and giving patients regular opportunities to provide feedback about the services they receive.

Table 2.

Patients’ Perceptions at T2 Regarding Changes in Patient Engagement at the Forensic Hospital.

			Valid % of respondents
	N	M (SD)	Not at all (1)	Slightly (2)	Somewhat (3)	Moderately (4)	Extremely (5)
Since our last interview, how much has the forensic hospital improved at . . .
Valuing and considering patients’ choices and preferences about treatment decisions	24	3.00 (1.32)	20.8%	12.5%	20.8%	37.5%	8.3%
Giving patients an opportunity to be involved in the services and supports they receive	22	2.82 (1.18)	9.1%	36.4%	31.8%	9.1%	13.6%
Providing patients with education and support so that they can take an active role in managing their own illness and taking care of themselves	23	2.74 (1.42)	26.1%	21.7%	17.4%	21.7%	13.0%
Giving patients regular opportunities to raise issues, questions and concerns regarding the services they receive	25	2.48 (1.33)	28.0%	32.0%	12.0%	20.0%	8.0%

We hypothesized that patients who had invested more time in the intervention would perceive greater improvements with respect to patient engagement at the forensic hospital. To test this hypothesis, we trisected the participants into three groups (low, moderate, and high) according to the distribution of their attendance in the intervention and compared their average scores across four domains of patient engagement that are listed in Table 2. The only significant difference was concerning perceptions that patients had been given greater opportunity to be involved in services and supports. Ratings for this domain differed significantly between patients whose level of participation in the intervention was low (n = 9, M = 2.00, SD = 0.71), moderate (n = 5, M = 3.40, SD = 1.14), or high (n = 8, M = 3.38, SD = 1.19), F(2, 21) = 5.10, p < .05. Tukey’s post hoc analysis revealed that patients with low levels of participation in the intervention rated this domain significantly lower than patients with moderate (p = .05) or high (p < .05) levels of participation.

We then examined perceived improvements within each of the intervention components (i.e., peer support, PAC, and Team PEER). Participants who were involved in the PAC perceived greater improvement, compared with those who did not participate in PAC, in the area of giving patients opportunities to be involved in services and supports at the forensic hospital, t(20) = 2.26, p < .05. A similar finding was detected when comparing the average ratings on this item for patients who did or did not receive peer support, t(20) = 2.16, p < .05. Because of variability in the frequency of receiving peer support, further analyses were performed on participants who had more (n = 10) or less (n = 15) than five sessions of individual- and/or group-based peer support. Patients who received five or more peer support sessions perceived greater improvements with involving patients in services, t(20) = 2.81, p = .01, and giving patients opportunities to raise issues, questions, and concerns regarding services, t(23) = 2.43, p < .05.

Table 3 summarizes the ratings of service providers regarding their perceptions of the patient engagement intervention at T2.

Table 3.

Service Providers’ Perceptions at T2 Regarding Changes Produced by the Patient Engagement Intervention.

			Valid % of respondents
	N	M (SD)	Not at all (1)	Slightly (2)	Somewhat (3)	Moderately (4)	Extremely (5)
During the past year, how much has . . .
Peer support program improved patients’ experiences	17	3.82 (1.24)	0%	23.5%	11.8%	23.5%	41.2%
PAC improved patients’ experiences	21	3.43 (1.25)	9.5%	14.3%	19.0%	38.1%	19.0%
Team PEER improved patients’ experiences	17	3.71 (0.99)	0%	11.8%	29.4%	35.3%	23.5%
Patient-centered care been improved	20	3.20 (1.15)	15.0%	5.0%	30.0%	45.0%	5.0%

Note. PAC = Patient Advisory Committee; Team PEER = Team Patients Empowered and Engaged as Researchers.

The majority of the service providers indicated that the patients’ experiences in the forensic hospital had been “moderately” or “extremely” improved as a function of the PAC (57.1%, n = 12), Team PEER (58.8%, n = 10), or peer support (64.7%, n = 11). Overall, 50.0% (n = 10) of the service providers had observed “moderate” to “extreme” improvements with respect to patient-centered care during the past year in the forensic hospital. Only 15.0% (n = 3) indicated that they had not observed any improvements in relation to patient-centered care.

Qualitative findings

Table 4 summarizes the themes from the participants’ qualitative responses about the patient engagement intervention.

Table 4.

Qualitative Themes About the Benefits Produced by the Patient Engagement Intervention as Perceived by Patients and Service Providers.

Intervention component	Thematic area	Illustrative quotes
Peer support program	Promoted positive role modeling	“I think it’s [peer support program] been a positive experience because it’s nice to know that some people who have mental illness are still able to contribute to the society.” (Patient)
	Created openness and emotional support	“Yes it [peer support program] does help me! . . . It helps me get out what I’m feeling inside.” (Patient)
	Gave hope and encouragement	“It [peer support service] was uplifting. . . . Just someone easy to talk to. Someone who listened, that was an active listener. I felt heard, understood.” (Patient)
PAC	Produced empowerment	“Patients now feel they have a voice.” (Service provider)
	Created community	“[I look forward to] seeing all those people that belong to the committee, and getting together and talking about all the issues.” (Patient)
	Fostered communication	“I’m glad to be a part of it [PAC]. . . . It fosters good communication in staff and patients.” (Patient)
	Cultivated change	“They [the PAC] get things done. Obviously the favorite thing that I got out of it was that we got coffee in the mornings . . . so, it has had an impact.” (Patient)
Team PEER	Increased knowledge and awareness	“[I] learned a lot about the hospital and how the hospital functions and surrounding around treatment teams. . . . So it was a great learning experience.” (Patient)
	Fostered a sense of pride and empowerment	“For the patients that have been directly involved, it [Team PEER] has really improved their experiences. Many of the patient members have told me . . . that they feel empowered.” (Service provider)
	Created camaraderie	“I think that there’s been a real camaraderie of team members that’s revolved around the project and I’ve enjoyed all that.” (Patient)

Note. PAC = Patient Advisory Committee; Team PEER = Team Patients Empowered and Engaged as Researchers.

From their perspective, peer support was effective at promoting positive role modeling, creating avenues for openness and emotional support, and giving patients hope and encouragement. The narratives about the PAC suggested that it empowered patients, created a sense of community among patients, fostered communication, and brought about patient-initiated improvements at the forensic hospital. The participants also identified the ways in which Team PEER improved patients’ experiences, particularly for the peer researchers, by increasing knowledge and awareness about the operations of the hospital, fostering a sense of pride and empowerment among patients, and creating camaraderie by working together on a research project.

Discussion

The current study evaluated the effectiveness of an intervention for (a) supporting forensic inpatients to participate in a range of patient engagement activities, (b) improving their experiences of care in the hospital, and (c) producing positive outcomes for patients as well as the broader service system. The intervention involved providing patients with opportunities to participate in peer support, patient advisory roles, and peer research. Changes in personal recovery, internalized stigma, personal empowerment, service engagement, therapeutic milieu, and recovery-oriented care were measured using standardized measures administered to patients and service providers at the beginning and end of the intervention. Additional quantitative and qualitative data were gathered to assess how the intervention was perceived by patients and service providers in the forensic hospital.

The evaluation demonstrated that forensic inpatients can be supported to participate in a diverse range of engagement activities, including peer support, peer research, and patient advisory roles. Implementing these strategies in a forensic hospital required careful planning and continuous problem solving, which was made possible by working with a team comprised of senior leaders and other staff with different roles within the hospital. Each of these patient engagement strategies required significant staff time and resources to ensure that patients received adequate support, monitoring, and mentoring in order for them to take on peer support, peer research, or patient advisory roles. The need to allocate sufficient resources for staff to lead and support the different components of the intervention was one of the lessons that we learned and should be taken into consideration for future patient engagement initiatives.

Although few significant positive changes were found on the standardized outcome measures, the intervention did improve the experience of care for patients. The majority of patients who participated in the study perceived some improvements with respect to patient engagement in the forensic hospital. For instance, two thirds indicated that positive changes had been achieved in the area of valuing and considering patients’ choices and preferences about treatment decisions, and more than half reported that patients were being provided with more opportunity to be involved in services and supports. Patients also indicated in the qualitative data that there were many ways that their involvement in the patient engagement initiative improved their experiences of care, such as improving feelings of hope and empowerment, and creating a sense of community and camaraderie. This suggests that the intervention did have important impacts that were not captured by the quantitative outcome measures.

The study revealed that patient participation in the peer support program, but not the other two components of the intervention, was associated with increases in personal recovery and may have prevented increases in levels of internalized stigma. This can be partially explained by the fact that the peer support program was the only component of the patient engagement intervention to have therapeutic support as its main objective. Other elements of peer support, such as role modeling and shared lived experience, may also have contributed to the results of this study. Our findings are consistent with the existing research demonstrating that peer support can produce improvements across a range of recovery-related outcomes for people with mental illness (Pistrang et al., 2008; Repper & Carter, 2011; Verhaeghe, Bracke, & Bruynooghe, 2008; Wright-Berryman et al., 2011).

The divergence between the quantitative and qualitative findings may reflect the methodological limitations of the evaluation. With a relatively small sample size, the study was underpowered for detecting small to medium effects using t tests, but had sufficient power for detecting medium to large effects. This was confirmed by performing post hoc power analyses using G*Power version 3.1 (Faul, Erdfelder, Buchner, & Lang, 2009), which revealed that the power for detecting small (d_z = 0.20), medium (d_z = 0.50), and large (d_z = 0.80) effects was 0.16, 0.67, and 0.97 for patients, respectively, and 0.11, 0.41, 0.79 for service providers, respectively. The naturalistic, observational design of the current study created some limitations since participants were not randomly allocated to study or control groups, and the dosage of patient engagement was not controlled. Additionally, it may be possible that the types of changes produced by the intervention were not adequately captured by the standardized measures or were confounded by external factors. For instance, the positive effects produced by the intervention may have been overshadowed by broader forensic context, in which patients can feel disenfranchised by the involuntary processes and legal mechanisms that preside over decisions regarding their care and liberties, including discharge from hospital. In forensic settings, it may be that greater momentum and demonstrable improvements in patient engagement efforts will have to be attained before desired results will be evident on existing metrics. Fortunately, our mixed method approach went beyond standard metrics to provide a more nuanced and complex picture with respect to the impact of patient engagement intervention in the forensic hospital.

It is also possible that the design of the intervention was, as a whole, ineffective for improving outcomes such as personal empowerment or the perceived recovery orientation of services. Although we had hypothesized that our engagement strategies would result in observable gains, there is little, if any, literature on the dose or extent of engagement activities that are required to produce measurable improvements—certainly not in forensic settings. Since forensic patients are vulnerable to practices of real or perceived coercion, it is likely that demonstrable desired results will not be achieved by simply adopting patient engagement strategies from other health care contexts. We suspect that forensic settings require patient engagement strategies that are relatively more robust and longer-term to counteract the pernicious effects that involuntary services can have on forensic patients’ level of trust and perceptions of care (Livingston, Rossiter, & Verdun-Jones, 2011). Although our study demonstrated that moderate exposure to peer support in a forensic hospital can positively influence internalized stigma and personal recovery, it is important to test and evaluate a range of patient engagement strategies that may produce the same or better results. Indeed, the question of which patient engagement activities are most effective for achieving certain outcomes remains unclear. Further research elucidating the linkage between patient engagement (e.g., type, amount) and a range of outcomes is urgently needed to provide useful guidance to service providers and to inform the development of effective, evidence-based engagement strategies.

On completing the evaluation, we were left with some unresolved questions regarding how to effectively involve a broader range of patients and service providers in patient engagement activities. The intervention focused entirely on creating organizational structures to engage with patients; however, we noticed that our major challenge was engaging and educating staff, particularly front-line nurses, about the importance of patient engagement and recovery-oriented care approaches. This barrier was exacerbated by not having a robust body of evidence on which we could say firmly that patient engagement would produce better mental health outcomes. Moreover, mental health service provision in a forensic inpatient hospital—where there is a real potential for dangerous situations to occur—involves fear and requires caution and vigilance (Jacob, Gagnon, & Holmes, 2009; Livingston et al., 2012; Mason, 2002). Therefore, front-line service providers may perceive strategies that promote power sharing with patients as being risky and, consequently, may be unwilling to support or participate in such activities. Further research is needed to better understand the influences that patient engagement approaches have on the safety and security of forensic and other mental health contexts. Investing resources into training front-line service providers about the principles and practices of patient engagement, patient-centered care, recovery, and related topics is likely to be an important first step toward achieving widespread improvements in patient engagement.

Conclusion

Patients in a forensic mental health hospital can be supported to participate in a range of engagement activities, including peer support, peer research, and patient advisory roles. It is likely that people living with mental illness, particularly those who have restrictions on their liberties, will respond positively to the increased opportunities to get involved in their own care. Encouraging patients to participate in the design, delivery, and evaluation of services in a forensic hospital may improve experiences of care. Further research is needed to investigate the dose, range, and effects of patient engagement on forensic mental health outcomes.

Footnotes

Author Roles

Dr. Livingston was involved in the conception, design, and implementation of the study, as well as the analysis and interpretation of the data and writing of the article. Ms. Nijdam-Jones was involved in the design and implementation of the study, as well as the collection and interpretation of the data. Ms. Lapsley and Ms. Calderwood were involved in interpreting the data. Dr. Brink was involved in the conception, design, and implementation of the study, as well as the interpretation of the data. All authors contributed toward writing the article and providing editorial support.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Financial support for this study was provided by the Canadian Foundation for Healthcare Improvement and BC Mental Health & Addiction Services.

References

Bullock

W. A.

(2005). Mental health recovery measure. In Campbell-Orde

Chamberlin

Carpenter

Leff

H. S.

(Eds.), Measuring the promise of recovery: A compendium of recovery measures (Vol. II, pp. 133-135). Cambridge, MA: Human Services Research Institute.

Cayton

(2004). Patient engagement and patient decision-making in England. London, England: Department of Health for England, The Commonwealth Fund/Nuffield Trust. Retrieved from http://www.commonwealthfund.org/~/media/Files/Publications/FundReport/2004/Mar/PatientEngagementandPatientDecisionMakinginEngland/cayton_patientengland_735pdf.pdf

Corlett

Miles

(2010). An evaluation of the implementation of the recovery philosophy in a secure forensic service. British Journal of Forensic Practice, 12(4), 14-25.

Corrigan

P. W.

Roe

Tsang

H. W.

(2011). Challenging the stigma of mental illness: Lessons for therapists and advocates. Malden, MA: Wiley.

Crawford

M. J.

Rutter

Manley

Weaver

Bhui

Fulop

Tyrer

(2002). Systematic review of involving patients in the planning and development of health care. British Medical Journal, 325, 1263. doi:10.1136/bmj.325.7375.1263

Faul

Erdfelder

Buchner

Lang

A.-G.

(2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149-1160. doi:0.3758/BRM.41.4.1149

Forbat

Hubbard

Kearney

(2009). Patient and public involvement: Models and muddles. Journal of Clinical Nursing, 18, 2547-2554. doi:10.1111/j.1365-2702.2008.02519.x

Geller

J. L.

(2012). Patient-centered, recovery-oriented psychiatric care and treatment are not always voluntary. Psychiatric Services, 63, 493-495. doi:10.1176/appi.ps.201100503

Green

Batson

Gudjonsson

(2011). The development and initial validation of a service-user led measure for recovery of mentally disordered offenders. Journal of Forensic Psychiatry & Psychology, 22, 252-265. doi:10.1080/14789949.2010.541271

10.

Gregory

(2007). Conceptualising consumer engagement: A review of the literature. Retrieved from http://www.healthissuescentre.org.au/documents/items/2009/06/280548-upload-00003.pdf

11.

Gudjonsson

G. H.

Webster

Green

(2010). The recovery approach to care in psychiatric services: Staff attitudes before and after training. The Psychiatrist, 34, 326-329. doi:10.1192/pb.bp.109.028076

12.

Hanley

Bradburn

Barnes

Evans

Goodare

Kelson

Wallcraft

(2004). Involving the public in NHS public health, and social care research: Briefing notes for researchers. Hampshire, England: INVOLVE Support Unit.

13.

Hillbrand

Young

J. L.

Griffith

E. E. H.

(2010). Managing risk and recovery: Redefining miscibility of oil and water. Journal of the American Academy of Psychiatry and the Law, 38, 452-456.

14.

Jacob

J. D.

Gagnon

Holmes

(2009). Nursing so-called monsters: On the importance of abjection and fear in forensic psychiatric nursing. Journal of Forensic Nursing, 5, 153-161. doi:10.1111/j.1939-3938.2009.01048.x

15.

Le Boutillier

Leamy

Bird

V. J.

Davidson

Williams

Slade

(2011). What does recovery mean in practice? A qualitative analysis of international recovery-oriented practice guidelines. Psychiatric Services, 62, 1470-1476.

16.

Liberman

R. P.

Kopelowicz

(2005). Recovery from schizophrenia: A concept in search of research. Psychiatric Services, 56, 735-742. doi:10.1176/appi.ps.56.6.735

17.

Livingston

J. D.

Boyd

J. E.

(2010). Correlates and consequences of internalized stigma for people living with mental illness: A systematic review and meta-analysis. Social Science & Medicine, 71, 2150-2161. doi:10.1016/j.socscimed.2010.09.030

18.

Livingston

J. D.

Nijdam-Jones

Brink

(2012). A tale of two cultures: Examining patient-centered care in a forensic mental health hospital. Journal of Forensic Psychiatry & Psychology, 23, 345-360. doi:10.1080/14789949.2012.668214

19.

Livingston

J. D.

Nijdam-Jones

Team

P. E. E. R.

(2013). Perceptions of treatment planning in a forensic mental health hospital: A qualitative, participatory action research study. International Journal of Forensic Mental Health, 12(1), 42-52. doi:10.1080/14999013.2013.763390

20.

Livingston

J. D.

Rossiter

K. R.

Verdun-Jones

S. N.

(2011). “Forensic” labelling: An empirical assessment of its effects on self-stigma for people with severe mental illness. Psychiatry Research, 188(1), 115-122. doi:10.1016/j.psychres.2011.01.018

21.

Mason

(2002). Forensic psychiatric nursing: A literature review and thematic analysis of role tensions. Journal of Psychiatric and Mental Health Nursing, 9, 511-520. doi:10.1046/j.1365-2850.2002.00521.x

22.

McEvoy

Keenaghan

Murray

(2008). Service user involvement in the Irish health services: A review of the evidence. Retrieved from http://www.hse.ie/eng/services/Publications/Your_Service,_Your_Say_Consumer_Affairs/Reports/LiteratureReview.pdf

23.

McLoughlin

K. A.

Geller

J. L.

Tolan

(2011). Consider this . . . Is recovery possible in a forensic hospital setting? Archives of Psychiatric Nursing, 25, 390-391.

24.

Mead

Bower

(2000). Patient-centredness: A conceptual framework and review of the empirical literature. Social Science & Medicine, 51, 1087-1110. doi:10.1016/S0277-9536(00)00098-8

25.

Mezey

G. C.

Kavuma

Turton

Demetriou

Wright

(2010). Perceptions, experiences and meanings of recovery in forensic psychiatric patients. Journal of Forensic Psychiatry & Psychology, 21, 683-696. doi:10.1080/14789949.2010.489953

26.

Mullen

P. E.

(2000). Forensic mental health. British Journal of Psychiatry, 176, 307-311. doi:10.1192/bjp.176.4.307

27.

Neergaard

M. A.

Olesen

Andersen

R. S.

Sondergaard

(2009). Qualitative description—The poor cousin of health research? BMC Medical Research Methodology, 9, 52. doi10.1186/1471-2288-9-52

28.

Nilsen

E. S.

Myrhaug

H. T.

Johansen

Oliver

Oxman

A. D.

(2010). Methods of consumer involvement in developing healthcare policy and research, clinical practice guidelines and patient information material. Cochrane Database of Systematic Reviews. doi:10.1002/14651858.CD004563.pub2

29.

O’Brien

White

Fahmy

Singh

S. P.

(2009). The development and validation of the SOLES, a new scale measuring engagement with mental health services in people with psychosis. Journal of Mental Health, 18, 510-522. doi:10.3109/09638230902968225

30.

O’Connell

Tondora

Croog

Evans

Davidson

(2005). From rhetoric to routine: Assessing perceptions of recovery-oriented practices in a state mental health and addiction system. Psychiatric Rehabilitation Journal, 28, 378-386. doi:10.2975/28.2005.378.386

31.

Pistrang

Barker

Humphreys

(2008). Mutual help groups for mental health problems: A review of effectiveness studies. American Journal of Community Psychology, 42, 110-121. doi:10.1007/s10464-008-9181-0

32.

Repper

Carter

(2011). A review of the literature on peer support in mental health services. Journal of Mental Health, 20, 392-411. doi:10.3109/09638237.2011.583947

33.

Resnick

S. G.

Rosenheck

R. A.

(2008). Integrating peer-provided services: A quasi-experimental study of recovery orientation, confidence, and empowerment. Psychiatric Services, 59, 1307-1314. doi:10.1176/appi.ps.59.11.1307

34.

Ridley

Jones

(2002). User and public involvement in health services: A literature review. Edinburgh, Scotland: Health and Social Care Directorates. Retrieved from http://www.sehd.scot.nhs.uk/involvingpeople/Aliteraturereview.pdf

35.

Ritsher

J. B.

Otilingam

P. G.

Grajales

(2003). Internalized stigma of mental illness: Psychometric properties of a new measure. Psychiatry Research, 121(1), 31-49.

36.

Rogers

E. S.

Chamberlin

Ellison

M. L.

Crean

(1997). A consumer-constructed scale to measure empowerment among users of mental health services. Psychiatric Services, 48, 1042-1047.

37.

Schalast

Redies

Collins

Stacey

Howells

(2008). EssenCES, a short questionnaire for assessing the social climate of forensic psychiatric wards. Criminal Behaviour and Mental Health, 18(1), 49-58. doi:10.1002/cbm.677

38.

Sidani

(2008). Effects of patient-centered care on patient outcomes: An evaluation. Research & Theory for Nursing Practice, 22(1), 24-37. doi:10.1891/0889-7182.22.1.24

39.

Simces

(2003). Exploring the link between public involvement/ citizen engagement and quality health care. Ottawa, Ontario, Canada: Zena Simces. Retrieved from http://www.hc-sc.gc.ca/hcs-sss/alt_formats/hpb-dgps/pdf/pubs/2003-qual-simces/2003-qual-simces-eng.pdf

40.

Simmons

Hawley

C. J.

Gale

T. M.

Sivakumaran

(2010). Service user, patient, client, user or survivor: Describing recipients of mental health services. Psychiatrist, 34(1), 20-23. doi:10.1192/pb.bp.109.025247

41.

Simpson

A. I. F.

Penney

S. R.

(2011). The recovery paradigm in forensic mental health services. Criminal Behaviour and Mental Health, 21, 299-306. doi:10.1002/cbm.823

42.

Tambuyzer

Pieters

Van Audenhove

(2011). Patient involvement in mental health care: One size does not fit all. Health Expectations. doi:10.1111/j.1369-7625.2011.00743.x

43.

Tempier

Hepp

S. L.

Duncan

C. R.

Rohr

Hachey

Mosier

(2010). Patient-centered care in affective, non-affective, and schizoaffective groups: Patients’ opinions and attitudes. Community Mental Health Journal, 46, 452-460.

44.

van Gestel-Timmermans

Brouwers

E. P.

van Assen

M. A.

van Nieuwenhuizen

(2012). Effects of a peer-run course on recovery from serious mental illness: A randomized controlled trial. Psychiatric Services, 63(1), 54-60. doi:10.1176/appi.ps.201000450

45.

Verhaeghe

Bracke

Bruynooghe

(2008). Stigmatization and self-esteem of persons in recovery from mental illness: The role of peer support. International Journal of Social Psychiatry, 54, 206-218. doi:10.1177/0020764008090422

46.

Warner

(2010). Does the scientific evidence support the recovery model? The Psychiatrist, 34(1), 3-5. doi:10.1192/pb.bp.109.025643

47.

Wright-Berryman

J. L.

McGuire

A. B.

Salyers

M. P.

(2011). A review of consumer-provided services on assertive community treatment and intensive case management teams: Implications for future research and practice. Journal of the American Psychiatric Nurses Association, 17(1), 37-44. doi:10.1177/1078390310393283