Development of a ‘real-world’ logic model through testing the feasibility of a complex healthcare intervention: the challenge of reconciling scalability and context-sensitivity

Abstract

Logic models feature prominently in intervention research yet there is increasing debate about their ability to express how interventions work in the real-world. ‘Real-world’ logic models are a new proposition which express complex interventions in context. They are designed to help researchers strike a balance between context-sensitivity and scalability. This article explores the utility of real-world logic models in a trial involving a complex intervention called ‘Your Care Needs You’, designed to improve hospital-home transitions for UK older patients. The approach is found to usefully capture, refine and express important learning about intervention-implementation-context dynamics. The findings imply the need for intervention researchers to think creatively about how to implement interventions in diverse and sometimes challenging environments and to develop understanding of how complex interventions adapt on implementation to produce outcomes. The possibility of assessing the wider social and policy context within intervention research is also posed.

Keywords

complex intervention complexity science context health services research implementation science logic model

Background

Discourses of evidence-based healthcare have undergone considerable evolution since emerging in the 1980s. Central to evidence-based healthcare is intervention research, whereby interventions for improving patients’ health or an aspect of health service performance are designed and evaluated for widespread adoption and scale-up (Sidani, 2014). Early guidelines for deriving and assessing evidence within intervention research posited a fixed, evidence-based hierarchy, with the Randomised-Control Trial (RCTs) situated at the top (Ho et al., 2008). This initial version of intervention research was complemented, during the mid 2000s, by implementation science, which sought to address a perceived evidence-practice gap by developing scientifically proven techniques for implementing interventions in practice. Implementation science recognises that the real-world differs markedly from the controlled conditions of RCTs, shifting the focus of research to how interventions work ‘in context’ (Rhodes and Lancaster, 2019). Yet, debate is ongoing about whether this twinned-approach, of scientifically evaluating interventions first and leaving tricky questions about how to implement interventions in the real-world to after efficacy has been established, underestimates the challenges involved in developing and utilising evidence in healthcare delivery (Braithwaite et al., 2018; Greenhalgh and Papoutsi, 2018; Rhodes and Lancaster, 2019).

Early versions of evidence-based healthcare are widely criticised for failing to recognise that healthcare interventions are qualitatively different to easily standardisable, clinical interventions (Ho et al., 2008). This criticism has even been levelled at attempts, emerging in the late 2000s, to permit a greater role for qualitative research in intervention research (Cohn et al., 2013; Greenhalgh and Papoutsi, 2018). Indeed, the Medical Research Council’s (MRC) ‘Developing and Evaluating Complex Interventions’ promoted theory-based approaches to intervention development and evaluation, including the use of logic models, for understanding the causal processes through which complex interventions produce outcomes (Craig et al., 2008). Critics argue that the MRC guidance has a reductionist concept of complexity that sees it as a property of interventions alone rather than the delivery settings into which they are placed. Complex interventions are defined as containing several interacting components (Craig et al., 2008; Moore et al., 2015). More recently, some researchers have developed complex interventions that feature adaptive components that ‘flex’ depending on features of the delivery setting, so potentially taking on a different form each time they are delivered (Greenhalgh and Papoutsi, 2018; Hawe, 2015; Mills et al., 2019a). This capacity to adapt on delivery enhances the potential for effectiveness in specific sites through self-organisation and sustainability (Hawe, 2015).

Two contrasting perspectives have emerged that present different visions for evidence-based healthcare in an age of complex, adaptive interventions. A radical position rejects the dominant epistemological and ontological assumptions of evidence-based healthcare, in particular, its ‘product-oriented focus’ (Bonell et al., 2012; see also Greenhalgh and Papoutsi, 2018; Rhodes and Lancaster, 2019). Scholars associated with this position question the very notion of an intervention as something that is designed external to delivery settings, with potential for scientific evaluation and scalability: Cohn et al. (2013), for example, critique evidence-based healthcare’s commitment to ‘the generalisability and reproducibility of things’. Similarly, a shift is advocated in how context is theorised and practically encountered in health research. The people, relationships, culture and local knowledge that feature in the contexts of research projects are viewed positively as assets to be utilised. Experimental methodologies such as the RCT are eschewed in favour of formative, participatory methods that allow ideas to emerge, or be adapted from elsewhere, through active participation guided by real-time feedback (Chambers et al., 2013).

The radical, participatory ‘turn’ in healthcare research promises uniquely context-sensitive solutions yet it is not without its critics. A key weakness is the time and effort it takes to design and implement innovations through participatory research (Bush et al., 2017). Given resource constraints, some form of scale-up of successful innovations is vital to continually improve patient care. A further weakness is that the drivers of impact can be unclear in participatory research, further inhibiting potential for scale-up. The people and relationships formed, as well as the evaluation processes that provide real-time data to inform decision-making, have causal, outcome implications in their own right. Participatory research tends, however, not to reflect on how these would have to be replicated alongside the changes enacted if success were to be scaled up (Hammersley, 2004; Mills et al., 2019b). Similarly, implementation strategies do not feature in participatory research because of the absence of a coherent intervention for delivery across different settings. A clear and evidence-based implementation strategy, which can itself be tested and refined through scientific evaluation, is vital if successful actions in one setting are to be replicated at scale (Curran et al., 2012).

In contrast to the radical position, scholars more sympathetic to the initial aims of evidence-based healthcare, of achieving research impact through widespread adoption and scale-up, have sought to incorporate context-sensitivity into the paradigm while retaining the possibility of scientific evaluation. Realist methodology has been influential here, although there is ongoing debate as to whether RCTs are consistent with realist principles (van Belle et al., 2016). In an early contribution, Penny Hawe sought to more directly close the circle between scalability and context-sensitivity by reconceptualising standardisation within trial settings to allow adaptations to occur (Hawe, 2004). Traditionally, interventions have been standardised by form and their delivery has been tightly prescribed. Hawe, in contrast to this traditional approach, recommends standardising by function so there is flexibility in what is delivered. Local actors can then be involved in shaping the intervention’s form, so enhancing local ownership and sustainability (Hawe, 2015). The distinction between common and variable components (Lilford, 2018) allows for similar flexibility.

In sum, current discourses of evidence-based healthcare exhibit a tension between scalability and context-sensitivity, which is expressed in Figure 1. At point A in the graph is the dominant model of evidence-based healthcare, which presupposes relatively standardisable or ‘complicated’ interventions, affords priority to RCT evidence and sharply distinguishes between intervention research and implementation science. At point C in the graph is the radical, participatory turn which rejects the RCT and even the desirability of evidence-based healthcare’s aim of achieving success at scale. At point B in the graph is an emerging field of research that aims to close the circle between scalability and context-sensitivity. Central to this emerging field is a notion of complex, adaptive interventions that are standardised in terms of function and thus can be tested in trials despite variation in what is delivered. Despite this conceptual innovation, however, there are a few published examples of trials of complex, adaptive interventions. This article contributes to the emerging research field around point B through a grounded example of a trial involving a complex, adaptive intervention, called Your Care Needs You. The article focuses on the use of a new logic model approach, called Real-World Logic Model (RWLM), to develop understanding of Your Care Needs You, as part of the feasibility stage of the trial.

Figure 1.

Trade-off in evidence-based healthcare discourses.

The ‘Your Care Needs You’ intervention

‘Your Care Needs You’ is a complex intervention developed as part of the Partners-at-Care-Transitions programme of research, a multi-stage, National Institute for Health Research (NIHR)-funded study of transitions from hospital to home for older patients (Baxter et al., 2020; Baxter et al., 2018; O’Hara et al., 2019, 2020). A theory-based approach to intervention development was adopted which sought to accommodate context-sensitivity within a trial setting. Context-sensitivity was achieved in the following two key ways: first, the initial programme theory that guided intervention development was developed on the basis of extensive exploratory research of the problem of poor transitions while – second – though various ‘fixed’ patient-facing intervention components were developed in light of the learning acquired, the plan going into the feasibility stage of the trial was for these to be delivered alongside flexible, staff-led improvement work that would adapt to the priorities and characteristics of each ward setting.

Figure 2 presents the programme theory (Funnell and Rogers, 2011; Oosthuizen and Louw, 2013) for the Your Care Needs You intervention prior to the feasibility study. The exploratory research that was undertaken prior to Your Care Needs You’s design revealed various activities, or ‘functions’, that patients perform once they get home that are key to a successful transition. These functions are the following:

Managing health and wellbeing;

Managing medications;

Completing daily activities;

Appropriately escalating care if needed.

Within the hospital setting, however, hospital staff are typically focused on discharging patients from hospital rather than preparing patients for performing these functions once they arrive at home. The research team sought to develop an intervention that would shift this focus and enhance how patients are supported and prepared in hospital to retain capacity and capability for the key functions they undertake once home. Hence, various patient-facing components were developed to activate the patient role in hospital. These included a Your Care Needs You booklet containing key information that is useful for patients and carers to know, along with some question prompts they may like to ask of staff, which is provided to patients within the hospital setting. A Your Care Needs You video and poster display, also shown in the hospital setting, reinforce the importance of patients and carers taking on a more active role in hospital when that is appropriate. Once their hospital stay is over, patients are then provided, at the point of discharge, enhanced discharge documents which contain patient-friendly information about what has happened in hospital, what patients can expect going forward and who to contact should they need to escalate care. These patient-facing components aim to help patients develop the knowledge and skills for taking on the four functions when they return to the community, leading to improved outcomes in the long term (see Figure 2).

Figure 2.

Linear logic model of the Your Care Needs You intervention.

The research team recognised early on that staff support would be vital if the patient-facing Your Care Needs You components were to successfully activate an enhanced patient role in hospital as only staff can, for example, address patients’ questions about medications or support patients’ desire to move about in the hospital setting. Staff would therefore have to be informed about and trained in how to deliver and support Your Care Needs You, implying a significant role for both staff and implementation teams who train and support them. Furthermore, staff would be encouraged, as part of the intervention, to enhance what they do to support patients prepare for home alongside the delivery of the patient-facing components. Various staff-facing components, including handouts and leaflets, were therefore developed to encourage staff to consider how they might enhance what they currently do to support the functional areas (i.e. health and wellbeing, medications, daily activities, care escalation). The idea was for staff to come up with ideas and implement changes, with implementation teams supporting their efforts using Quality Improvement (QI) techniques. This staff-level improvement work is a variable component of Your Care Needs You, in contrast to the fixed, or ‘common’ (Lilford, 2018) patient-facing components. This combination of common and variable components aligns Your Care Needs You with the emerging field of research around point B of Figure 1.

The feasibility of Your Care Needs You (including the patient- and staff-facing components) was explored in the feasibility study over a period of 3 months, along with an implementation strategy featuring four key components: first, a 1- to 2-hour meeting, chaired by implementation teams, would enable Multi-Disciplinary Teams (MDT) to adapt Your Care Needs You to their ward setting; second, staff roles in supporting the patient-facing components and the staff-led improvement work would be assigned locally; third, brief staff training would equip staff with the knowledge and skills required to support the intervention; fourth, a ‘share and learn’ session would identify and resolve implementation problems at the halfway point. Exactly how the various intervention components and implementation strategy would work, however, as well as the precise roles of and inter-relationships between patients, staff and implementation teams, was unclear at the start of the feasibility study. Hence, the research team decided to develop a RWLM to facilitate a more comprehensive understanding of Your Care Needs You than the initial, linear logic model provides.

Real-world logic models (RWLM)

The RWLM approach was developed in response to a debate, taking place as the discourses of evidence-based healthcare have evolved, about whether logic models can provide a context-sensitive account of how interventions work (Hawe, 2015; Funnell and Rogers, 2001; Mills et al., 2019a). Mills et al (2019a) reviewed this debate in their article ‘Advancing complexity science in healthcare research: the logic of logic models’ and proposed RWLM as a way forward for the paradigm.

Mills et al (2019a) outline various principles for intervention researchers to consider when creating logic models and present a flexible template for complex, adaptive interventions, based on a model of a patient experience toolkit intervention that was developed as part of a participatory research project (see Figure 3). Intervention researchers are encouraged to adapt the template through the creative use of shapes and arrows to accommodate contingencies and dynamic relationships. The template features a ‘core’ for describing the intervention and the people involved. Descriptions can encapsulate activities or mechanisms or, ideally, combinations of the two, as a focus on activities can be reductionist but mechanisms can be overly abstract. The key features of context that shapes intervention delivery and outcomes, or ‘moderators’, are listed above and below this core, while the various types of outcome (proximal/distal and core/peripheral) are situated to the right. In a conducive context (one in which the moderators take the form of enablers for implementation teams to utilise rather than barriers), intervention activities should lead to the core outcomes; in an unconducive context (one in which one or more moderators are experienced as a barrier), implementation teams must work to negate or modify the context if the intervention is to succeed.

Mills et al. (2019a) found RWLM to enrich participatory research as it provides a multi-level framework for understanding interventions undergoing development and a guide for achieving scale-up outside of initial research settings. In the case of the RWLM for the patient experience intervention expressed in Figure 3, for example, implementation teams can use it to assess new delivery settings and adapt what they do to implement the intervention accordingly. For example, where the moderators of staff engagement or self-efficacy are perceived as being high, implementation teams should utilise these to develop and implement ambitious changes to patient services; where they are low, implementation teams may have to build morale by focusing on ‘quick wins’ or ‘celebrating successes’ first before aiming for more substantial service improvements (Mills et al, 2019a; Mills et al., 2019b; Sheard et al., 2019). This context-sensitive facilitation work can produce soft or peripheral outcomes which can, in turn, ease implementation and make the core outcomes more easily attainable, as implied by the two-way arrows linking the core and the peripheral outcomes in Figure 3. Of course, new settings will present new challenges such that RWLM will only ever be an imperfect guide but they should themselves be subject to ongoing revision so that new learning can be captured. As the RWLM approach was developed as part of a participatory research project, however, it is unclear whether it is appropriate in a trial setting. This article’s assessment of RWLM in a trial of the Your Care Needs You intervention therefore represents an important contribution to the RWLM paradigm and complex intervention research more generally.

Figure 3.

A RWLM of a patient experience toolkit intervention.

Methodology

The utility of RWLM was assessed during the feasibility stage of a trial of the Your Care Needs You intervention. 10 diverse hospital wards were involved in the study across four NHS Hospital Trusts, with intervention wards (n = 6, though two wards did not deliver the intervention due to severe staff and service pressures) testing the Your Care Needs You intervention and control wards (n = 4) delivering care as usual over a 3-month period. As the Your Care Needs You intervention is designed for older patients, wards where at least 40 per cent of patients are aged 75 and above were included. It was anticipated that intervention wards would deliver the Your Care Needs You intervention to all patients aged 75 or above who were expected to go home after their hospital stay, or their carers or families. The feasibility study protocol has been published (Baxter et al., 2020) and a full findings paper, reporting on the feasibility of trial methodology and the Your Care Needs You intervention, will be published shortly. This article reports on the development of a RWLM as part of the qualitative research, with the aim being to enrich understanding of how the Your Care Needs You intervention is to work in the trial.

The first step to develop a RWLM of the Your Care Need You intervention was to incorporate aspects of the linear model (see Figure 2) into the template provided by Mills et al (2019a; see Figure 3). The research team then created lists of possible intervention and implementation mechanisms, moderators and outcomes to be tested and refined against the qualitative data collected in the study, in a form of abductive qualitative analysis (Lipscomb, 2012). Qualitative data were collected mainly through semi-structured interviews and observation (Lambert and McKevitt, 2002). Members of the research team went onto each participating ward at least twice a week to recruit patients into the trial. They were instructed to undertake ward-level observations while on the wards, with a focus on what happened; staff and patient responses to, and views of, the Your Care Needs You intervention; and barriers and enablers to implementation. Authors 1 and 2, performing the dual role of researchers and implementers, also undertook ward-level observations while working closely with ward staff to implement the intervention. In addition, author 1, author 2 and two research nurses kept reflective diaries to record meta-level observations based on their experiences of working across the wards and hospitals involved. In total, 91 discrete ward-level observations were written by the research team and the reflective diaries contained 41 pages of reflection. Ten patients were also recruited for detailed patient-level observations and follow-up interviews once they were discharged, while 17 staff, each of whom had a key role in the project, were recruited for interview. Topic guides for both staff and patient interviews focused on the feasibility of the intervention but flexibility was permitted so that discussions could explore emergent findings. Some interviews were recorded but environmental factors on the wards meant this was not always possible. Researchers wrote up detailed accounts of each interview on their completion. The research team also had frequent meetings as data were being collected during which they interpreted emerging findings and discussed how the Your Care Needs You intervention and implementation strategy would be refined ahead of the trial. Notes were kept of these meetings.

The data analysis phase involved the development of a coding framework that encapsulated all project data. Authors 1 and 2 developed the coding framework together, analysing parts of the data in isolation before comparing their interpretations and resolving any differences through consensus discussion. They then summarised the codes into themes, using thematic analysis techniques (Braun et al., 2018). The RWLM for Your Care Needs You was developed alongside the coding process, with the RWLM template being adapted for Your Care Needs You and the lists of possible mechanisms, moderators and outcomes being tested and refined for inclusion in the model. Both the themes and the RWLM were frequently shared with the wider research team for comments. 3x planning sessions were held to reflect on the findings and plan the implementation strategy for the full trial, with the RWLM being revised to capture what was planned. The final RWLM, presented below, reflects the shared expectation of the research team of how the intervention and its implementation strategy will function in the trial.

Findings

Data pertaining to the feasibility of Your Care Needs You and the implementation strategy were summarised into four key themes. These provide background insight into what happened on the hospital wards and staff and patient views of the intervention. The RWLM for the Your Care Needs You intervention is presented after the themes and a narrative explains each part of the RWLM and how it reflects the themes. A summary of the research team’s observations of the utility of RWLM in a trial setting is then presented.

Themes

Theme 1: Patients generally like the Your Care Needs You intervention but staff input is crucial and efforts to involve patients must be sensitive to their health states, capacity and preferences

Patients and carers mostly liked the intervention although the Your Care Needs You booklet was rarely used to communicate with staff directly, with patients preferring to read it and write notes in it instead. Indeed, two patients spoke of how it had legitimised a more questioning role for them to take on while a carer had not realised, until reading the booklet, that hospital staff might know detailed information about what happens after the hospital stay; they planned to ask more questions in future. While patients sometimes acted on the booklet unprompted and this could remind staff about the intervention, they were more likely to do so when encouraged by staff and staff input was required for individual patient questions to be addressed. The input of both staff and patients is therefore vitally important and can reinforce each other, with staff input sometimes making up for a lack of patient input and vice versa. Patients did, however, differ in terms of their capacity and motivation to read and use the Your Care Needs You booklet and there was some concern, shared among patients and staff, that the booklet could induce additional stress and anxiety for patients who lack capacity, or if it is introduced too soon in their hospital stay.

Theme 2: A Multi-Disciplinary Team for supporting intervention delivery is vital but difficult to sustain due to staff rotation

The greatest success occurred on a ward where the project had the backing of a senior clinician who galvanised staff of diverse roles (both clinical and non-clinical) to support the intervention. Staff reported enhanced discharge processes, with conversations about discharge occurring earlier on during patient stays. This activity proved short-lived however, as the clinician rotated and both implementation teams and even ward managers found it difficult to engage senior clinicians at ward-level in general. In the absence of a functioning MDT, the project sometimes came to be something for one individual or one staff group to do. Individuals or particular staff groups could become resentful if they felt they were being asked to do too much while the absence of diverse skill sets weakened the intervention’s potential; for example, therapy staff led on the intervention on one ward which meant patients had support in their efforts to be active in hospital but the absence of nursing and pharmacy staff meant they did not have support practicing medications.

Theme 3: Acute staffing pressures and a lack of clarity regarding staff roles can skew attention to the patient-facing components rather than on how staff can enhance their support for patients

Service and staffing pressures were ever-present and complicated the intervention’s delivery on all intervention wards. In an extreme case, one hospital did not deliver the intervention at all because they could not free up staff time to participate, meaning that two intervention wards did not deliver the intervention. In general, moreover, the hour-long, implementation meeting with staff focused on the patient-facing components and how staff might support patients’ use of them, such that the flexible, staff-level component of Your Care Needs You was sidelined. Staff reported being confused about their role in it. Staff on one ward did create a poster for patients about a physio-led exercise class they offered as part of this flexible improvement work while another ward sought to encourage patients, with staff or carers supporting, to visit a nearby outdoor balcony that featured some seats with a view of the local city. Due to the challenges involved with embedding the patient-facing components, however, these initiatives were introduced towards the end of the feasibility study and there were limited opportunities for implementation teams to help implement them through the application of QI techniques, as was originally planned.

Theme 4: Extensive, adaptive facilitation by implementation teams is required to ensure Your Care Need You’s sustained success

Service pressures and staff rotation necessitated frequent visits by implementation teams to support and train staff. The staff-facing components which included handouts and leaflets were helpful here. Information about post discharge risks appeared to resonate with staff who saw it as part of their role to prepare patients for home, although implementation teams observed that staff-facing components did not provide clear information about the flexible, staff-led improvement work. As well as their facilitation at ward-level, implementation teams worked across host organisations to overcome barriers that ward teams were experiencing. This included, for example, obtaining laptops for staff to show the Your Care Needs You intervention and seeking permissions from senior management to allow carers onto wards following an organisation-wide freeze on visits due to an outbreak of norovirus on a ward that was not involved in the project. In one hospital, the local research and development (R&D) team provided extensive assistance to implementation teams in this facilitation work and an idea to involve volunteers to help deliver the patient-facing components (so freeing up the time of frontline staff to enhance what they do to prepare patients for home) was discussed but not implemented due to the feasibility study ending.

RWLM for Your Care Needs You

The feasibility study provided the opportunity to test and develop the initial programme theory for the Your Care Needs You intervention which, as expressed in the linear logic model above (see Figure 2), hinged on the activation of patients in hospital who, by acquiring more skill and knowledge in that setting through the use of the patient-facing components, are predicted to experience improved outcomes at home. The RWLM of Your Care Needs You, presented in Figure 4, provides a more comprehensive, in context account of this complex, adaptive intervention; the circles and ovals in the model represent dynamic relationships between factors contained within them while the boxes represent a more distal relationship with intervention activities. Each part of the RWLM is explained in turn.

Figure 4.

A RWLM of the Your Care Needs You intervention.

Mechanisms

The feasibility study confirmed the expectation that the patient-facing components would be insufficient on their own to enhance patient preparations for home, with staff input required to support their use (see Theme 1). Similarly, implementation teams have an equally vital role in intervention delivery, including both ‘upward escalation’ across organisations and ‘downward support’ to staff at ward-level. Hence, the ‘core’ in the centre of the RWLM for Your Care Needs You includes each of these parties (implementation teams, staff and patients). The central role of implementation teams has been described, in the RWLM, as being to create opportunities for staff to help patients prepare for home. This reflects how implementation teams understood what they were doing and will ensure that Your Care Needs You’s delivery does not become the sole responsibility of staff. Whether opportunities are created or not shapes the opportunities open to patients to prepare for home, thus setting the conditions for the enhanced patient role in hospital and improved, post-discharge outcomes. The activation of both staff and patient roles also requires bringing to their attention the risks of a lack of patient preparedness, although this must be done sensitively and take into account patient preferences, health states and capacity (see Theme 1).

The feasibility study also served to clarify the role of the patient-facing components in the intervention and, based on the learning acquired, they were retheorised as ‘boundary objects’ (Star and Griesemer, 1989) to reflect how their use must adapt to each patient and also how they have different purposes and meanings to the various parties involved. As part of this retheorisation, the staff-facing components – which aim to inform staff about Your Care Needs You – have also been amended, ahead of the trial, to provide clearer roles for how staff can support patients going home to avoid staff focusing on the delivery of the patient-facing components alone (see Theme 3).

Implementation strategy

As it became clear just how significant the implementation strategy is to Your Care Needs You’s success (see Theme 4), the RWLM was expanded to incorporate the key activities of the strategy, situated in the box and the circle to the left of the core. The box features aspects of the strategy that are relatively detached while the circle is used, both here and for the proximal outcomes, to reflect their dynamic relationship with core activities; for example, the training and facilitation that implementation teams provide, including the upwards escalation to overcome barriers experienced or their downwards support to ward teams, must adapt to what is happening on each ward, as it is central to creating the opportunities for staff to help patients prepare for home. The separation of the implementation strategy from the core, despite a close relationship between the two, represents a change from the initial RWLM template which incorporated the implementation strategy as part of the intervention (through the role of the facilitator; see Figure 3). It will enable evaluators in the trial to assess the implementation strategy in its own right, alongside the Your Care Needs You intervention.

Due to resource constraints, implementation teams will not be available to provide the level of facilitation and support in the trial that they provided here because the number of wards will be greater. As part of the implementation strategy, therefore, implementation teams will work across organisations to form a Trust-wide Your Care Needs You team that they will work with to support staff at ward-level. Some R&D teams were effectively performing this role in the feasibility study (see Theme 4). The Trust-wide Your Care Needs You team will include senior figures from host organisations, local improvement teams whose role is to improve services through the use of Quality Improvement techniques, and Principal Investigators, although precise roles and responsibilities will adapt to capacity and capability. Recruiting non-ward staff, including corporate personnel and hospital volunteers, to undertake some of the basic tasks of intervention delivery would also help to free up the time of frontline staff so that they have greater opportunity to focus on helping patients prepare for home and avoid a restrictive focus on the patient-facing components (see Theme 3).

Context

Whereas the initial RWLM template utilised the distinction between inner and outer features of context, the RWLM for the Your Care Needs You intervention includes the micro, meso and macro features of context. This reflects differences in the underlying research paradigms, as the initial template was designed as part of a participatory research project. It therefore had a practical emphasis such that only factors that could be utilised or required some modification by implementation teams were included while broader policy and socioeconomic factors (i.e. the macro-level) were omitted. By contrast, the macro-level is included here to enable evaluators of Your Care Needs You to assess the full context of the trial. Macro-level factors feature in a box at the top of the model, rather than an oval, to reflect how they have a more distal relationship with intervention activities while still being experienced at lower levels.

Both the implementation strategy and intervention activities will be enabled and constrained in the full trial by the factors listed across micro-, meso- and macro-levels; implementation teams and staff will therefore have to adapt their approach accordingly. Implementation teams will adapt the level of support and facilitation they provide to the availability of and support from hospital personnel, described as part of the meso context in the RWLM. Strategies for overcoming barriers experienced in the micro-context will be implemented. This may include, for example, seeking positive stories from patients about their discharge experiences to share with staff to encourage engagement with the intervention. Finally, implementation teams will train staff in how to respond to variation in patient preferences and capacity by, for example, introducing the patient-facing components at the appropriate time in their stay or bypassing the patient-facing components altogether where there is clear reluctance to use it.

Outcomes

The distal outcomes are the measurable trial outcomes which are – at this stage – hypothesised as resulting from Your Care Needs You’s successful delivery and use. They are presented in a box rather an oval to represent their status as distal outcomes. The proximal outcomes are situated in a circle to convey their dynamic relationship with core activities, with Trust-wide Your Care Needs You teams being encouraged to capture and convey positive impacts of the intervention to staff to facilitate engagement. Unlike the RWLM template, the Your Care Needs You model does not include the peripheral outcomes of the project that arise from the context-sensitive facilitation work carried out by implementation teams. These have been omitted because space was not available and trial methodology assigns greater significance to core project outcomes than participatory research.

The RWLM of Your Care Needs You in the trial

The research team found the RWLM to usefully guide the capture of learning about intervention-implementation-context dynamics in the feasibility study. The model will enhance the implementation and evaluation of Your Care Needs You going forward. Possible uses of the RWLM in the full trial include a guide for implementation teams, as they go about setting up Trust-wide Your Care Needs You teams across host organisations and supporting and training ward teams in intervention delivery. Here, the RWLM’s utility stems from its inclusion of key features of context that will enable and constrain the work of implementation teams and which, if encountered, may require ameliorative action. The RWLM will also frame the intervention’s evaluation, including both indicators for outcome evaluation and insight into the key intervention and implementation mechanisms that evaluators will need to assess to evaluate the quality of implementation and the veracity of the initial programme theory underpinning Your Care Needs You (see Figure 2).

The trial evaluation, guided by the RWLM, will also assess the conduciveness of the macro-context to Your Care Needs You and continue to develop insight into the intervention’s interaction with factors across micro-, meso- and macro-levels. For example, the finding that Your Care Needs You can potentially legitimise a more questioning role for patients in busy hospital wards, discussed in Theme 1, is evidence of the mechanism of enhanced role congruence being activated. This is potentially significant and requires further substantiation because it implies a synergistic relationship with, and a contribution to, changing social norms about the patient role at the macro-level. In addition, while the implementation strategy for the trial has been adapted to take into account staff and service pressures, by including corporate staff and volunteers in intervention delivery, whether these measures are enough to modify or negate such a stubborn, macro-level barrier will be assessed. Such a comprehensive, multi-level evaluation will be important to accurately interpret trial outcomes and to inform robust post-trial recommendations; for, there may be a case for refining parts of the intervention and/or the implementation strategy once the trial is complete, or their present state may be optimal but current staff and service pressures are too great on hospital wards for the intervention’s effective delivery.

Discussion

Researchers, in health and other sectors, face a tension between context-sensitivity and scalability in their efforts to develop, implement and evaluate interventions for addressing complex problems. While trial methodology is sometimes seen to be at odds with the increasingly widespread recognition of the importance of context-sensitivity in research (Bonell et al., 2012; Cohn et al., 2013; Rhodes and Lancaster, 2019), complex, adaptive interventions balance the need for commonality across different sites while permitting local adaptations within trials. This article has shown that the RWLM approach, which has already been shown to help researchers extrapolate interventions from participatory research for wider scale-up (Mills et al., 2019a; Mills et al., 2019a; Sheard et al., 2019), has utility in a trial setting. Here, its utility derives from its capacity to provide a comprehensive, in context account of interventions that can help researchers to understand the combined function and effects of interventions, implementation strategies and context within trials. The utility of the approach is particularly apparent in the later stages of the intervention design and evaluation process, with RWLM providing a framework to capture and express intricate detail of how interventions work in context as they are undergoing testing in real-world conditions. A number of important implications for intervention research follow.

The Your Care Needs You case suggests that theory-based approaches to intervention research, which have hitherto focused primarily on intervention logics, need to encapsulate the broader array of factors (pertaining to interventions, implementation strategies and delivery settings) which shape intervention effectiveness. While a simple programme theory was suitable for the early stages of Your Care Needs You’s development, the primary mechanism of that programme theory (i.e. the activation of an enhanced patient role) was shown, in the feasibility study, to depend on a host of additional mechanisms, most importantly the success of implementation teams at creating the opportunity for staff to prepare patients for home. The implementation strategy was also central to Your Care Need You’s adaptation to its delivery settings, through the setting up of local support teams and the training of ward staff to take into account patients’ unique circumstances, preferences and capacity on intervention delivery. This capacity of RWLM to capture broader factors shaping effectiveness, as interventions are being tested in real-world conditions, makes RWLM an important complement to established programme theory (Funnell and Rogers, 2011; Oosthuizen and Louw, 2013) and theory of change approaches (Breuer et al., 2018) in intervention research.

The Your Care Needs You case also has significant implications for emerging literatures on complex interventions (around point B of Figure 1). The complex intervention literature proposes novel ways of thinking about standardisation that enable complex interventions to be evaluated in trials, notably function or form (Hawe, 2015) and common or variable (Lilford, 2018). The research team found these to be useful for theorising and permitting the flexible roles of the various components of Your Care Needs You. The Your Care Needs You case does, however, suggest that intervention researchers need to think creatively about how to implement complex interventions in diverse and sometimes challenging environments. Implementation strategies were shown to be central to complex interventions because they shape the specific forms that complex interventions take on and thus ensure that adaptations remain congruent with intervention aims; this crucial role of implementation strategies is under recognised in the complex intervention literature. RWLM provide a framework for capturing the details of implementation strategies for future implementation teams to emulate or adapt them across other settings. As with their use in participatory research, they can aid scale-up efforts by facilitating ever-expanding insight into relevant features of context that implementation teams must either utilise, modify or negate (Mills et al., 2019a). Yet, since new settings inevitably present new challenges and opportunities, they can only constitute guides for later scale-up efforts. RWLM should, therefore, be interpreted flexibly and be open to ongoing reiteration and development such that new learning can be incorporated.

The RWLM of Your Care Needs You used the micro, meso and macro framework to provide a sense of the levelling of different factors and the points at which they emanate or exert influence from. While two-dimensional models of the interaction between complex interventions and context will inevitably be somewhat blunt, this levelled approach can help intervention researchers to identify and make sense of interactions across levels. In the Your Care Needs You case, for example, the intervention was observed to legitimise a more questioning patient role. Although this outcome is likely to be moderated by patient characteristics at the micro-level, it potentially indicates a contribution, however small, to changing norms and practices at the system-level which entrench a more passive role. There were, however, limitations to the extent that macro-level factors that were experienced at micro- and meso-levels could be modified or negated at those levels. Staff resources, for example, represented a particularly stubborn barrier which shaped micro-level staff factors, notably staff engagement.

Indeed, in the Your Care Needs You case, as well as the patient experience study expressed in the initial RWLM, research teams and staff were required to creatively work around staffing pressures on the hospital wards by, for example, enacting less ambitious (and thus less time consuming) service improvements. Progress was not always possible and sometimes wards had to pull out of the studies altogether (Mills et al., 2019b; Sheard and Peacock, 2019; Sheard et al., 2019). Health services in the United Kingdom, along with other labour-intensive public services such as policing and social services (Lewis et al., 2019), are funded by central government. As such, staff and service pressures in these sectors constitute a macro-level, political issue and were frequently framed as such by participants of the study reported here. Yet, health research typically operates at the micro-level of individuals and teams or the meso-level of organisations, with multi-level analysis a rarity (Fulop and Robert, 2015). By contrast, the RWLM approach encourages researchers to consider the conduciveness of the wider policy and socioeconomic environment to interventions being delivered. This is vitally important as it provides a basis to highlight stubborn macro-level barriers when interpreting suboptimal intervention outcomes and to link evaluations to wider debates about government priorities and public sector funding. In this way, RWLM align with Okwaro et al.’s call for intervention research to stop bracketing out the political and economic realities that shape healthcare systems and ask ‘bigger picture questions’ (Okwaro et al., 2015).

Conclusion

RWLM can help researchers extrapolate, out of their research, scalable but context-sensitive interventions. They provide a multi-level framework for capturing and expressing learning about intervention-implementation-context dynamics that is particularly relevant and useful when interventions are being tested in real-world conditions. RWLM can be used to frame evaluations of complex interventions, or to guide scale-up efforts occurring outside of initial research settings. They also encourage intervention researchers to theorise interventions within the broader social, economic and political environment in which they are situated, providing a framework for identifying and making sense of interactions across micro-, meso- and macro-levels.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship and/or publication of this article.

Ethical approval

Approvals were gained from the Wales Rec 7 Research Ethics Committee, Confidentiality Advisory Group and the Health Research Authority prior to starting the study (REC reference 19/WA/0162, CAG reference 19/CAG/0105). Local NHS capability and capacity approvals were granted by all participating NHS Trusts. Any amendments to the protocol will be submitted for the required regulatory approval. The study is registered on the UK Clinical Research Network Study Portfolio (42191) and the ISTCRN (ISRCTN51154948).

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship and/or publication of this article: This report is independent research funded by the National Institute for Health Research (NIHR; National Institute for Health Research Programme Grants for Applied Health Research, Partners at Care Transitions (PACT)): Improving patient experience and safety at transitions in care, RP-PG-1214-20017. The views expressed in this publication are those of the author(s) and not necessarily those of the NIHR or the Department of Health and Social Care.

ORCID iD

Thomas Mills

Thomas Mills (PhD) is a health services and public health researcher based at the London South Bank University. They have expertise in qualitative research, process evaluations and complex interventions.

Rosie Shannon (MSc) is a health services researcher with expertise in applied, health intervention research, including the use of diverse mixed-methods approaches in hospital settings.

Jane O’Hara (PhD) is a Professor of Healthcare Quality and Safety at the University of Leeds. They are interested in the innovative use of theory and robust methods in patient safety research.

Rebecca Lawton (PhD) is a Professor of Psychology at the University of Leeds. They are a founder member of the Yorkshire Quality and Safety Group and their research interests span patient safety and lifestyle behaviour change.

Laura Sheard (PhD) is an Associate Professor at the York Trials Unit, based at the University of York with expertise in qualitative methods, process evaluations and health sociology.

References

Baxter

Murray

O’Hara

, et al. (2020) Improving patient experience and safety at transitions of care through the Your Care Needs You (YCNY) intervention: A study protocol for a cluster randomised controlled feasibility trial. Pilot and Feasibility Studies 6: 123.

Baxter

O’Hara

Murray

, et al. (2018) Partners at care transitions: Exploring healthcare professionals’ perspectives of excellence at care transitions for older people. BMJ Open 8: e022468.

Bonell

Fletcher

Morton

, et al. (2012) Realist randomised controlled trials: A new approach to evaluating complex public health interventions. Social Science and Medicine 75: 2299–2306.

Braithwaite

Churruca

Long

, et al. (2018) When complexity science meets implementation science: A theoretical and empirical analysis of systems change. BMC Medicine 16: 63.

Braun

Clarke

Hayfield

, et al. (2018) Thematic analysis. In: Liamputtong

(ed.) Handbook of Research Methods in Health Social Sciences. Singapore: Springer, 1–18.

Breuer

de Silva

Lund

(2018) Theory of change for complex mental health interventions: 10 lessons from the programme for improving mental healthcare. Global Mental Health 16: 5.

Bush

Pluye

Loignon

(2017) Organizational participatory research: A systematic mixed studies review exposing its extra benefits and the key factors associated with them. Implementation Science 12: 119.

Chambers

Glasgow

Stange

(2013) The dynamic sustainability framework: Addressing the paradox of sustainment amid ongoing change. Implementation Science 8: 117.

Cohn

Clinch

Bunn

, et al. (2013) Entangled complexity: Why complex interventions are just not complicated enough. Journal of Health Services Research & Policy 18(1): 40–43.

10.

Craig

Dieppe

Macintyre

, et al. (2008) Developing and evaluating complex interventions: The new Medical Research Council guidance. BMJ 337: a1655.

11.

Curran

Bauer

Mittman

, et al. (2012) Effectiveness-implementation hybrid designs: Combining elements of clinical effectiveness and implementation research to enhance public health impact. Medical Care 50: 217–226.

12.

Fulop

Robert

(2015) Context for successful quality improvement. The Health Foundation. Available at: www.health.org.uk

13.

Funnell

Rogers

(2011) Purposeful Program Theory: Effective Use of Theories of Change and Logic Models. San Francisco, CA: Jossey-Bass/Wiley.

14.

Greenhalgh

Papoutsi

(2018) Studying complexity in health services research: Desperately seeking an overdue paradigm shift. BMC Medicine 16: 95.

15.

Hammersley

(2004) Action research: A contradiction in terms? Oxford Review of Education 30(2): 165–181.

16.

Hawe

(2004) Complex interventions: How ‘out of control’ can a randomised controlled trial be? BMJ 328(7455): 1561–1563.

17.

Hawe

(2015) Lessons from complex interventions to improve health. Annual Review of Public Health 36: 307–323.

18.

Peterson

Masoudi

(2008) Evaluating the evidence is there a rigid hierarchy? Circulation 118: 1675–1684.

19.

Lambert

McKevitt

(2002) Anthropology in health research: From qualitative methods to multidisciplinarity. BMJ 325: 210.

20.

Lewis

Higgins

Muir

(2019) Police workforce wellbeing and organisational development. The Police Foundation. Available at: www.police-foundation.org.uk

21.

Lilford

(2018) Implementation science at the crossroads. BMJ Quality and Safety 27: 331–332.

22.

Lipscomb

(2012) Abductive reasoning and qualitative research. Nursing Philosophy 13(4): 244–256.

23.

Mills

Lawton

Sheard

(2019a) Advancing complexity science in healthcare research: The logic of logic models. BMC Medical Research Methodology 19: 55.

24.

Mills

Lawton

Sheard

(2019b) Improving patient experience in hospital settings: Assessing the role of toolkits and action research through a process evaluation of a complex intervention. Qualitative Health Research 29(14): 2108–2118.

25.

Moore

Audrey

Barker

, et al. (2015) Process evaluation of complex interventions: Medical research council guidance. BMJ 350: 1258.

26.

O’Hara

Aase

Waring

(2019) Scaffolding our systems? Patients and families ‘reaching in’ as a source of healthcare resilience. BMJ Quality & Safety 28: 3–6.

27.

O’Hara

Baxter

Hardicre

(2020) ‘Handing over to the patient’: A FRAM analysis of transitional care combining multiple stakeholder perspectives. Applied Ergonomics 85: 103060.

28.

Okwaro

Chandler

Hutchinson

, et al. (2015) Challenging logics of complex intervention trials: Community perspectives of a health care improvement intervention in rural Uganda. Social Science and Medicine 131: 10–17.

29.

Oosthuizen

Louw

(2013) Developing program theory for purveyor programs. Implementation Science 8: 23.

30.

Rhodes

Lancaster

(2019) Evidence-making interventions in health: A conceptual framing. Social Science & Medicine 238: 112488.

31.

Sheard

Peacock

(2019) Fiddling while Rome burns? Conducting research with healthcare staff when the NHS is in crisis. Journal of Health Organization and Management 34(1): 77–84.

32.

Sheard

Marsh

Mills

, et al. (2019) Using patient experience data to develop a patient experience toolkit to improve hospital care: A mixed-methods study. Health Services and Delivery Research 7(36).

33.

Sidani

(2014) Health Intervention Research: Understanding Research Design and Methods. London: SAGE.

34.

Star

Griesemer

(1989) Institutional ecology, ‘translations’ and boundary objects: Amateurs and professionals in Berkeley’s museum of vertebrate zoology, 1907-39. Social Studies of Science 19(3): 387–420.

35.

van Belle

Wong

Westhorpe

, et al. (2016) Can ‘realist’ randomised control trials be genuinely realist? Trials 17: 313.