PROTOCOL: Barriers and facilitators to stakeholder engagement in health guideline development: A qualitative evidence synthesis

The problem, condition or issue

Health guidelines are ‘systematically developed evidence-based statements which assist providers, recipients and other stakeholders in making informed decisions about appropriate health interventions’ (WHO, 2003). Guidelines play a crucial role in the delivery of evidence-based medicine. The systematic examination of evidence promoted by evidence-based medicine is done through a comprehensive search of the literature, critical appraisal of the quality of the evidence and interpretation of the findings in light of patients' preferences and societal values (Guyatt, 2008). These guidelines assist health practitioners, patients, caregivers, policymakers and other stakeholders to make informed decisions about health practice, public health and health policy. It is increasingly recognised that the engagement of multiple stakeholders in guideline development improves recommendation relevancy, uptake, implementation and sustainability (Esmail, 2015; Gagliardi, 2012; Moulding, 1999).

Stakeholders are ‘any individual or group who is responsible for or affected by health- and healthcare-related decisions that can be informed by research evidence’ (Concannon, 2012). This broad definition includes, but is not limited to, patients, caregivers, the public, patient advocates, providers (e.g., nurses), policymakers (e.g., health ministries), payers of health services (e.g., insurers, public payers) and health research (e.g., national research institutes), purchasers (e.g., employers), researchers, product makers (e.g., drug makers) and peer review editors. Several entities, including the World Health Organization (WHO), the Canadian Institute for Health Research (CIHR), the National Institute for Health and Care Excellence (NICE) and the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group recommend the inclusion of stakeholder groups in the guideline development and implementation process (Akl, 2017; CIHR, 2014; NICE, 2014; WHO, 2003). The Guidelines International Network (GIN) McMaster Checklist for Guideline Development, the global standard for guideline development, identifies ‘consumer and stakeholder involvement’ as Step 6 in the guideline development process (Schünemann, 2014).

There is a growing body of research into the methods for engaging stakeholders in guideline development (Armstrong, 2017; Cluzeau, 2012), as well as approaches to measuring the impact of stakeholder engagement in guideline development (Cottrell, 2014; Ray, 2017). For example, a 2017 review of the incorporation of patients' values and preferences in guidelines included guidance documents from 56 different organisations (Selva et al., 2017). However, there remains a need for the development of comprehensive, global, evidence-based guidance for stakeholder engagement in health guideline development that brings together the vast literature amassed to date. The Multi-Stakeholder Engagement Consortium (MuSE Consortium) was established in 2015 to advance methods and approaches used in stakeholder engaged health research. The consortium addresses research of all kinds—including reviews and guidelines – and includes more than 80 researchers, trainees and stakeholders from Australia, Brazil, Canada, Germany, Italy, Lebanon, the Netherlands, the Philippines, Switzerland, the UK and the USA. The team membership of MuSE includes researchers, policymakers, guideline developers, research funders, clinicians, patients, patient representatives and policymakers from various organisations including the Agency for Healthcare Research and Quality (AHRQ), the Campbell Collaboration, Cochrane, GRADE Working Group, Public Health Agency of Canada, Patient-Centered Outcomes Research Institute (PCORI), Research and Development (RAND) Corporation, the World Health Organization (WHO), Joanna Briggs Institute (JBI) and multiple universities (Petkovic, 2020). As part of the consortium's work, a working group was formed to develop global guidance on stakeholder engagement in guideline development according to the 18-step GIN-McMaster Checklist. The working group set out to conduct four concurrent systematic reviews to summarise the evidence on: (1) existing guidance for stakeholder engagement in guideline development, (2) barriers and facilitators to stakeholder engagement in guideline development, (3) managing conflicts of interest in stakeholder engagement in guideline development and (4) measuring the impact of stakeholder engagement in guideline development. This protocol addresses the second systematic review in the series.

Definitions

For the purpose of this review, we define:

Guidelinesas ‘systematically developed evidence-based statements which assist providers, recipients and other stakeholders in making informed decisions about appropriate health interventions’ (WHO, 2003).

Description of the phenomena of interest

Stakeholder engagement in guideline development is the approach to gather input or contribution from stakeholders ‘towards the development of a guideline, completion of any stages of a guideline, or dissemination, uptake or evaluation of a guideline and its recommendations’ (Pollock, 2018). The term ‘stakeholders’ is inclusive of multiple individuals within a single stakeholder group (such as several patients) and multiple stakeholders across several stakeholder groups (such as patients, policymakers and providers of care). Engagement refers to a meaningful and active partnership among stakeholder groups (CIHR, 2019; Staniszewska, 2017) to develop, complete or disseminate a guideline. Other terms for engagement include ‘public involvement’, ‘consumer engagement’, ‘Patient and Public Involvement (PPI)’, ‘coproduction’, ‘co-creation’, ‘activation’ and ‘involvement’.

Stakeholder engagement in research can occur at different levels of intensity, ranging from low-intensity communication (i.e., stakeholders are informed of research processes) up to high-intensity coproduction (i.e., stakeholders are equal members of the research team and participate in shared decision-making) (Concannon, 2012). From previous work (Crowe, 2017; Oliver, 2008; Pollock, 2019), we identify two levels of engagement in guideline development, namely (1) advisory/feedback, and (2) participation in decision-making or knowledge translation. These levels were selected for our operational use, but we acknowledge that within these, there may be different levels.

There are many activities required for effective stakeholder engagement in guideline development (see Figure 1). The engagement of stakeholders from multiple disciplines or specialities is typically recommended to take account of a wide range of opinions. Indeed, the greater diversity in stakeholders' background increases the knowledge and skills available to make decisions (Oliver, 2018). However, the ability to gather and apply input from multiple stakeholders in a way that meaningfully reflects and addresses stakeholders' interests is not easily achieved, as increased diversity may also reduce group cohesiveness (Oliver, 2018). In their review of theoretical literature, Brodbeck and colleagues (Brodbeck, 2007) proposed that groups can out perform individual decision makers under conditions which promote sharing of knowledge and mutual learning. Oliver and colleagues (Oliver, 2018) built on this model, and suggested that these conditions include diverse group membership, skilled facilitation, time for knowledge sharing, following formal consensus processes, and integrating divergent perspectives. Stakeholder engagement is a complex process, and achieving these conditions requires both individual and organisational adoption. Organisational cultures and relationships act as a backdrop to these stakeholder engagement activities (Oliver, 2018). There are a multitude of individual- and organisational-level factors that can affect a stakeholder's ability to engage in the guideline development process, or a guideline developer's ability to effectively engage them. Barriers to stakeholder engagement are defined as any variable, condition or factor that may impede stakeholder engagement in guideline development, while facilitators are any variable, condition or factor that may promote stakeholder engagement in guideline development. Such determinants may include knowledge, skills, time and resources, for example, and may occur across one or more steps of guideline development.

OPEN IN VIEWER

Figure 1

Logic model of the effects of stakeholder engagement in guideline development

Why it is important to do this review

With increased recognition of the value of stakeholder engagement in guideline development, there is a need for expert opinion and evidence-based guidance on when and how to engage stakeholders. This review of barriers and facilitators to stakeholder engagement in guideline development will help fill this gap. It will build on existing reviews of barriers and facilitators, synthesise the current evidence-base, and examine a broad range of stakeholder groups.

To date, several systematic reviews have been conducted on the topic of stakeholder engagement in guideline development, but few are explicitly focused on barriers and facilitators to engagement. Oxman and colleagues (Oxman et al., 2006) conducted a series of reviews to advise WHO to ensure that health care recommendations are informed by the best available research evidence. Relevant reviews in this series included group composition (Fretheim, 2006a), group processes (Fretheim, 2006b) and consumer involvement (Schünemann, 2006). However, these reviews were not systematic nor exhaustive, and prioritised existing syntheses over primary empirical studies. Relatedly, Oliver and colleagues (Oliver, 2018) synthesised theoretical literature to learn about the structure, processes and environment of committees generally, which has applications for health guideline development.

Much of the existing barrier and facilitator syntheses focus specifically on patients. Légaré and colleagues (Légaré, 2011) reviewed published and unpublished literature regarding the effectiveness of patient and public engagement programs in clinical practice guidelines and described the barriers and facilitators to these programs. Barriers included recruiting challenges, lack of representation of certain patient and public groups, and participants' lack of familiarity with medical and scientific terminology. Facilitators included the provision of training and support. Similarly, Grant and colleagues (Grant, 2018) conducted a rapid review and qualitative evidence synthesis to make inferences about how feasible online methods for patient engagement would be during clinical practice guideline development. The authors highlighted important considerations for patient and caregiver participation related to skills, time and resources. Notably, both of these reviews only considered patient and public involvement and did not consider the engagement of other stakeholder groups such as healthcare providers or policymakers, for example. Recently, Selva and colleagues (Selva, 2017) reviewed guidance documents for developing clinical guidelines to assess how these documents address the incorporation of patients' views. They identified the need for additional guidance in this area.

Cluzeau and colleagues (Cluzeau, 2012; Kelson, 2012; Kunz, 2012) conducted a literature review and workshop to answer six questions related to stakeholder engagement in guideline development, including identification of the potential barriers and facilitators to integrating stakeholder involvement. They defined stakeholders as ‘all those who have a legitimate interest in a guideline. They include healthcare professionals, patients and caregivers, public and private funding bodies, managers, employers and manufacturers’. They noted that integrating different, sometimes competing, stakeholder perspectives can be challenging. Other barriers included issues with potential bias among stakeholder views, and the costly nature of the process. In contrast, they found that educating stakeholders and ensuring effective communication enabled effective stakeholder engagement.

In sum, there is a lack of systematic reviews which have synthesised barriers and facilitators to multi-stakeholder engagement in health guideline development. The results of this review will be used to inform the development of guidance for multi-stakeholder engagement in guideline development and implementation. This guidance will be official GRADE (Grading of Recommendations Assessment, Development and Evaluation) Working Group guidance. The GRADE system is internationally recognised as a standard for guideline development (Guyatt, 2008). The findings of this review will assist organisations who develop healthcare, public health and health policy guidelines, such as the World Health Organization, to involve multiple stakeholders in the guideline development process to ensure the development of relevant, high quality and transparent guidelines.

Criteria for including and excluding studies

Search methods for identification of studies

This review is part of a series of four reviews conducted by the MuSE working group on stakeholder engagement in guideline development. As such, one comprehensive search strategy was developed and peer-reviewed in consultation with a medical librarian in accordance with PRESS guidelines (McGowan, 2016). A second medical librarian reviewed the search strategy.

Description of methods used in primary research

Relevant studies will have employed a qualitative or mixed-method design. For example:

Brouwers et al. (2017) (mixed-methods; Brouwers, 2017). Cancer patients, survivors, family members and caregivers were recruited to explore optimal approaches to patient and caregiver engagement in the development of cancer practice guidelines. Participants attended a workshop, completed a survey, or participated in a telephone interview. Quantitative data were analysed using Microsoft Excel, and qualitative data were analysed using thematic analysis. Authors report on barriers and facilitators to engaging in guideline development.

Criteria for determination of independent findings

Details of study coding categories

Data will be extracted by two review authors independently and in duplicate according to a standardised data extraction form (see Supporting Information Appendix 8). Disagreements will be discussed and resolved by a third reviewer. The data extraction form will be pilot tested by two review authors and coding categories will be clarified iteratively, as needed. Data will be extracted on basic study characteristics and methods including:

Study objective

In addition, we will extract information on study:

Limitations

For qualitative outcomes, we will extract data using codes adopted from the Theoretical Domains Framework (Atkins, 2017). We will extract stakeholders' perceptions and experiences regarding the barriers and facilitators to their engagement in health guideline development from the results and discussion sections, including participant quotations from interviews or focus groups, excerpts or quotations from documentary analysis, narrative descriptive summaries, author hypotheses, explanations and recommendations, themes and sub-themes.

Discrepancies in the data extraction process will be resolved by a third reviewer.

Critical appraisal

Two review authors will assess the methodological rigour of all included studies independently and in duplicate. We will assess the quality of qualitative studies and the qualitative components of mixed method studies using the Critical Appraisal Skills Programme (CASP) qualitative appraisal research tool (Singh, 2013). Discrepancies in the critical appraisal process will be resolved by a third reviewer.

Treatment of qualitative research

Certainty of evidence

We will use the Confidence in the Evidence from Reviews of Qualitative research (CERQual) tool to assess the confidence of our findings. This tool is a method for assessing the strength of qualitative review evidence, similar to how the GRADE approach assesses the strength of quantitative evidence (Lewin et al., 2018). CERQual bases the evaluation on four criteria: (a) methodological limitations of included studies supporting a review finding, (b) the relevance of included studies to the review question, (c) the coherence of the review finding and (d) the adequacy of the data contributing to a review finding. Final CERQual assessments will be presented as a CERQual Evidence Profile.