Barriers and Facilitators to Interest-Holder Engagement in Health Guideline Development: A Qualitative Evidence Synthesis

Types of Studies

We included primary research studies which qualitatively reported on the barriers or facilitators to interest-holder engagement in health guideline development. This included qualitative or mixed method research studies using methods such as interviews, focus groups, or surveys to collect participant experiences; case studies of existing programmes; and process evaluation studies.

We included published guidelines that reported lessons learned only if these statements were derived from empirical qualitative methods and the details of these methods are available in a published report. Mixed method and intervention studies (randomised or non-randomised designs) were eligible only if they qualitatively report barriers and facilitators to engagement, independent of their quantitative findings.

We excluded non-empirical publications including commentaries and editorials. We excluded publications with incomplete data, including conference abstracts and protocols. Finally, we excluded existing evidence syntheses (e.g., systematic reviews, meta-ethnographies, scoping reviews) as it would not be possible to extract the information relevant to our review, and reviewed their reference lists to identify any potentially eligible primary research articles.

Types of Participants

The population of interest is interest-holders in health guideline development. We define “interest-holders” as groups with legitimate interests in the health issue under consideration. The interests arise and draw their legitimacy from the fact that people from these groups are responsible for or affected by health-related decisions that can be informed by research evidence (Akl et al., 2024). Building on previous work, we have identified 10 types of interest-holders whose input can enhance the relevance and uptake of guidelines (Concannon et al., 2012, 2018; Tugwell, 2006). We included studies which reported on the barriers or facilitators to engagement of one or several of these interest-holder groups at any step of the guideline development process. For the purposes of this review, have grouped them as follows:

Patients, caregivers, and patient advocates

We recognise that within each of these categories, there is diversity and heterogeneity in types of interest-holders. For example, we recognise that patients and caregivers have differing characteristics and experiences, and their engagement brings diverse value to guideline development. The categories outlined above are for operational purposes and are based on prior research (Concannon et al., 2012, 2018; Tugwell, 2006). Studies which consider a single group of interest-holders or multiple groups of interest-holders were eligible for inclusion. If a study included multiple interest-holder groups, findings were coded for each interest-holder group separately.

Findings (Qualitative Data) of Interest

The primary findings of interest were any barriers and facilitators to interest-holder engagement in health guideline development. Barriers were defined as any variable or condition that impedes interest-holder engagement in health guideline development. Facilitators were defined as any variable or condition that promotes interest-holder engagement in health guideline development. To be eligible, these findings could be reported as first or second order constructs related to interest-holders’ and study team perceptions and experiences of barriers and facilitators to their engagement in guideline development. First-order constructs refer to original narratives (e.g., participant quotes and observations) in the reviewed articles that relate to a particular topic. The interpretations of these narratives by the authors of the articles (e.g., themes, author explanations or recommendations) are considered second-order constructs, while the synthesis of the review team are third-order constructs (Figure 3). In this review, both first order and second order constructs were considered eligible data for inclusion, extraction and analysis and could be located in an articles’ method, results, discussion or conclusion sections of the report.OPEN IN VIEWER

Setting

We did not place any restrictions on geographic settings. Studies focused on health guideline development from any country in the world were eligible for inclusion.

Analysis

Our review included studies employing a qualitative approach to inquiry. We used the ‘best fit’ framework method as a systematic and flexible approach to analysing the qualitative data (Booth & Carroll, 2015; Carroll et al., 2013; Gale et al., 2013). Framework-based synthesis, using the ‘best fit’ method, is a highly pragmatic and useful strategy for analysing a range of questions and is supported by guideline development literature (Dixon-Woods, 2011; Flemming et al., 2019). Framework analysis is a five-stage process that includes familiarisation with the data, identifying a thematic framework, indexing (applying the framework), charting and mapping, and interpretation (Ritchie & Spencer, 1994).

Data were analysed deductively, using the TDF domains to generate a framework of preliminary themes. Whilst the domains are purposefully designed to be broad groupings of the possible factors to influence behaviour, the intent is to explore the important domains in further detail. Two review authors (O.M. and A. Raymond) examined the data that were extracted under each TDF domain. The two authors identified and came to agreement on recurring concepts to develop the initial themes. Disagreements were resolved through discussion. When necessary, the authors returned to the full texts of the included studies to verify the perspective and context of the extracted data. After developing the initial thematic findings, all themes were circulated to the authorship team for comment. In a series of iterative drafts, the themes were refined and expanded upon. An illustrative quote was selected for each theme and the authorship team decided to organize themes using tabular and narrative approaches.

All identified key findings were mapped to the 18 topics of guideline development process outlined by the GIN-McMaster checklist (Schünemann et al., 2014). Through team discussions, we opted to remove topic 6 (Consumer and interest-holder engagement), as this project moves engagement from a stand-alone topic to a cross-cutting consideration for guideline developers to consider throughout each step of guideline development. This change is reflected in our published GIN-McMaster Guideline Development Checklist extension for engagement (Petkovic et al., 2025). An initial categorization was proposed by the lead author (O.M.) and then iteratively revised by the authorship team. We present this information as a narrative and series of tables, indicating the barriers and facilitators that exist for each interest-holder group and for each step of guideline development. The findings of this mapping exercise will be combined with the results of the other reviews in this series (Khabsa et al., 2022; Lytvyn et al., 2024; Petkovic et al., 2022) to develop international guidance on interest-holder engagement in guideline development.

A Lack of Human and Financial Resources at the Organisational Level Limited the Collaborative Development of Recommendations (CERQual: Moderate Confidence, n = 6 Studies)

Resource constraints were a general concern across several studies, which acted as a barrier to engaging patients, caregivers, patient organisations, and providers of care. Three studies specifically recognized the high level of human resources necessary to coordinate interest-holder engagement efforts (Den Breejen et al., 2014; Ilott et al., 2006; van Wersch & Eccles, 2001), citing the need for dedicated paid staff or coordinator positions to handle the logistics of consultations, which was particularly challenging for smaller organisations (Ilott et al., 2006). Specific engagement methods, such as surveys, interviews or focus groups, were identified as being desirable to capture the patient perspective in guidelines (M. J. Armstrong et al., 2020; Brouwers et al., 2017), but not all organisations would have the resources for such work (Hämeen-Anttila et al., 2016).

Guideline Development Organisations Have Unclear Goals Related to Patient and Public Engagement (CERQual: Very Low Confidence, n = 2 Studies)

Guideline developers and organisations often lacked clear goals about patient and public engagement, being described as “largely implicit or articulated in vague terms […] which makes it difficult to assess their success or failure” (Boivin et al., 2010). Patients who are engaged in guideline development may have different understandings of the goal of their involvement; in one study patients believed that their involvement was a mechanism to change their own healthcare and disclose personal health problems, which derailed the guideline development group from its true purpose (Lanza & Ericsson, 2000).

Guideline Developers Deliberately Exclude Patients From Guideline Development (CERQual: Low Confidence, n = 3 Studies)

Guideline developers may choose not to engage patients because they feel like the clinical practice guideline topic is too complex to require patient input (van de Bovenkamp & Zuiderent‐Jerak, 2013) or because they feel that patient contributions are too general for the question the specific guideline is trying to answer (Brouwers et al., 2018). Patient organisations do not always see the added value of participation either, as they may think that professionals already have the knowledge needed to develop a good guideline, and patients provide little added-value (van de Bovenkamp & Zuiderent-Jerak, 2013). In one study, guideline developers intentionally had no formal patient or public involvement to prevent tokenism (Selby et al., 2017).

Guideline Development Organisations Should Implement Formal Interest-Holder Engagement Models Into Their Everyday Operations (CERQual: Moderate confidence, n = 6 Studies)

To facilitate the engagement of providers of care, patients and caregivers, several professional bodies and organisations invested considerable resources in interest-holder relations. Clear, effective organisational leadership was an important enabler for interest-holder-engaged work (Walker et al., 2022). For example, incorporating stakeholding activities into their business plan and including these in the job description of professional guideline development coordinators (Ilott et al., 2006). This can facilitate engagement as the interest-holder engagement process becomes a part of the organisation’s everyday work. Guideline developers may also develop and incorporate engagement models into their guideline development (organisational) processes for future use (Ilott et al., 2006; van der Ham et al., 2014). This model sometimes results from patient and interest-holder engagement being a criterion to get development grants (van de Bovenkamp & Zuiderent‐Jerak, 2013).

Two engagement models have been evaluated: The Heart and Stroke Foundation of Canada developed an inclusive and integrated model of patient engagement named the Community Consultation and Review Panel (CCRP) (Lindsay et al., 2020). The goal of the CCRP model was to facilitate development of recommendations that are grounded in real-life experience and the unique needs of patients while maintaining scientific rigour. The model provided members with opportunities to regularly review and offer feedback on draft recommendations from the scientific writing group, and provided context related to their values and preferences (Lindsay et al., 2020). Similarly, a team of researchers from RAND and clinicians, caregivers, and patients from the Parent Project Muscular Dystrophy (PPMD) developed the RAND/PPMD Patient-Centeredness Method (RPM). The RPM is an online modified-Delphi approach to patient engagement that allows patients and their representatives to provide input on the patient centeredness of draft guideline recommendations (C. Armstrong et al., 2019). This model has been suggested for use throughout all stages of guideline development to continuously engage a panel of patients and their representatives (C. Armstrong et al., 2019).

Provide Interest-Holders With Financial Compensation And/Or Professional Accreditations (CERQual: Moderate Confidence, n = 9 Studies)

Guideline development organisations may facilitate interest-holder involvement by budgeting for and obtaining funds to financially compensate interest-holders, particularly for patient representatives as well as caregivers, patient organisations, and providers of care. This may include reimbursement for travel, printing, and other costs (e.g. childcare), as well as honoraria (financial support) to cover their time (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004). For providers of care, alternative compensation in the form of professional accreditations (e.g. CME) could be considered (Ilott et al., 2006). For all interest-holders, inclusion in publications if they meet the criteria for authorship should also be considered. One study suggested that covering expenses and providing compensation can facilitate engagement of patients/public from diverse identities (e.g. from different geographies, race/ethnicity, culture, language, etc.) (Synnot et al., 2022).

Guideline Developers Should Provide Training for Patient and Public Interest-Holders (CERQual: Moderate Confidence, n = 12 Studies)

Several studies highlighted the need for training or education and support for all interest-holders, but particularly patient and public members, to facilitate better understanding of the guideline development process (Biggane et al., 2022; Boivin et al., 2010; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Duff et al., 1996; Hämeen-Anttila et al., 2016; Jarret, 2004; van de Bovenkamp & Zuiderent‐Jerak, 2013). Training on how to effectively participate could be provided before and during involvement. A study by Daraz et al. (2019) found that pre-engagement training may be especially beneficial for patients as this can help assist with incorporating their input and preferences early into the guideline development process. They recommended that the training should be delivered prior to the first guideline panel meeting and before the final recommendation deliberation (Daraz et al., 2019). Training could include information about clinical guidelines, research methodology, evidence frameworks, and basic statistics. Daraz et al. (2019) described a training curriculum targeting patients’ knowledge, skills, and attitudes based on content from the Grading of Recommendations Assessment, Development and Evaluation (GRADE) Working Group and readability principles. In a webinar, the focus was on didactics and concepts, and in a workshop, the focus was on roleplay and practical issues (Daraz et al., 2019). Another study suggests that training could come in the form of orientation sessions by teleconference or in-person, by watching a video of a guideline development group in action, or through mentoring opportunities from other patient/public representatives (Jarret, 2004). Patient and public members should also be advised of various resources and support available and be provided with examples illustrating the impact of lay members in previous guideline developments (Biggane et al., 2022). Guideline development group members should be notified of the level of training and resources that patients receive before meetings (Brouwers et al., 2018). A study by Chartres et al. (2020) suggested that training all guideline development group members who are unfamiliar with evidence frameworks should be an ongoing process, and Armstrong et al. (2020) suggests this promotes co-learning.

Challenges Identifying “Representative” Patients to Engage in Guideline Development (CERQual: Low Confidence, n = 10 Studies)

Studies reported that identifying patients, caregivers and patient representatives to participate in clinical practice guideline development is a challenge (M. J. Armstrong et al., 2020; Biggane et al., 2022; Kim et al., 2021). There is some debate as to whether patients involved in guideline development should represent the larger patient population or simply provide their own unique patient perspective. In several studies, guideline development groups were uncertain about whether patients were being properly represented (Brouwers et al., 2018; Chalmers et al., 2017; Hämeen-Anttila et al., 2016; Lanza & Ericsson, 2000; van der Ham et al., 2014). One article specifically expressed concerns regarding patient proxies (such as caregivers or healthcare professionals) who may be included within a guideline development group to represent the patient’s interests, stating “interests and needs of patients [are] sometimes distinct to those of [other interest-holders]” (Biggane et al., 2022). Guideline development group members pointed out that it may be difficult to find appropriate persons from the target group who would be capable of representing the larger patient population. Consequently, balancing representation and interest-holder opinion was difficult in participatory development processes (Walker et al., 2022). In two studies, it was difficult to represent the diversity of the target group in the guideline development process with respect to ethnic diversity and health conditions (Kim et al., 2021; van der Ham et al., 2014). Armstrong et al. (2020) found that the experiences and perspectives of patients included in guideline development may be different from those who choose not to participate in the guideline development process, and emphasised that self-selection may occur and that diverse patient perspectives may be insufficiently articulated.

Patients Perceive ‘Tokenistic Engagement’ Which Limits Their Ability to Meaningfully Contribute to Guideline Development (CERQual: Low Confidence, n = 7 Studies)

Seven articles identified tokenistic engagement, that is, offering engagement as a gesture, as a challenge to meaningfully engaging patients (M. J. Armstrong et al., 2017; Atkins et al., 2013; Biggane et al., 2022; Brouwers et al., 2017; Daraz et al., 2019; Jarret, 2004; Murad et al., 2019). Instead of being equal members of guideline development groups, some patients expressed frustration and felt they were included merely to fill a quota (Biggane et al., 2022). This was described as being a “tokenized member of the team” rather than a fully integrated member (Brouwers et al., 2017) and highlighted the need for strategies to guard against tokenism, ensure that all members contributed equally to discussion and decision making, and ensure that the patient voice was adequately captured in the final guideline (M. J. Armstrong et al., 2017; Atkins et al., 2013).

Language Fluency May Limit the Representativeness of Patients and Their Ability to Engage in Group Processes (CERQual: Low Confidence, n = 3 Studies)

Three studies highlighted the potential for language and manner of speaking to become barriers to patient engagement (M. J. Armstrong et al., 2017; Chalmers et al., 2017; Piggott et al., 2021). For example, there may be a requirement for patients to speak only English in order to participate in guideline development meetings, leading to lack of representation from other linguistic groups (Chalmers et al., 2017). Community members (patients and the public) also expressed concern that physicians from other countries who are members of the guideline development group may have accents or speak differently, making it difficult for them to understand discussion (M. J. Armstrong et al., 2017). Interest-holders [guideline methodologists and content experts] have reported that different cultures and languages can have a major impact on group processes and that respect and consideration for this should be given by all guideline development group members (Piggott et al., 2021).

Patient/Public and Wider Interest-Holder Selection Should Consider the Representativeness and Diversity of Membership (CERQual: Low Confidence, n = 13 Studies)

Several studies mentioned the importance of recruiting ‘appropriate’ or ‘representative’ patients to participate in guideline development (M. J. Armstrong et al., 2017; Boivin et al., 2010; Brouwers et al., 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Harding et al., 2010; van der Ham et al., 2014; van Wersch & Eccles, 2001). Clear eligibility criteria is a facilitator to engaging people who represent the types of lived experience that are sought by the guideline developers; a “job description” may help to specify the knowledge, experience and social skills needed (Chalmers et al., 2017). Online modified-Delphi processes could be used to propose and prioritise desired characteristics of guideline development group members, including the type of expertise or experience group members should have (C. Armstrong et al., 2019). Involving multiple patient representatives or patient advocates may facilitate peer support and representation (Chalmers et al., 2017).

Project leaders may consider diversity as they are recruiting - this includes perspectives, roles, ethnicities, and other differences within the patient group. In order to properly consult marginalised patient sub-groups, it may be necessary to pay special attention to their inclusion. A study by Armstrong et al. (2020) found that local organisations could help to identify relevant patient representatives, as this could aid in overcoming recruitment difficulties as well as limit the use of resources required for their engagement. Using this recruitment approach, patient interest-holders had at least a basic knowledge of the relevant medical terminology (M. J. Armstrong et al., 2020). An additional article emphasised the importance of establishing respectful partnerships with organisations that work with or represent diverse groups, as they can assist with recruitment (Synnot et al., 2022). Power imbalances between members (e.g. between content experts and evidence review groups) could also potentially be minimised by paying attention to differences in age, experience, gender, or region and intentionally selecting members with diverse experiences and characteristics (Chartres et al., 2020). One study highlighted that special considerations would be needed to engage children in guideline development, as “clear requirements to give children a voice in the process of guideline development do not exist” (Schalkers et al., 2017).

The Recruitment of Patient Representatives Should Prioritise the Identification of Knowledgeable and Experienced Patient Representatives (CERQual: High Confidence, n = 6 Studies)

Many articles suggested that guideline development groups should recruit patients that are familiar with the guideline topic and process (M. J. Armstrong et al., 2017; Batalden et al., 2015; Brouwers et al., 2018; Hämeen-Anttila et al., 2016; Kim et al., 2021; van der Ham et al., 2014). Patient panellists should be knowledgeable, have relevant expertise, and be willing to actively contribute and ask questions (M. J. Armstrong et al., 2017). Several potential groups were identified – such as patient trainers, peer support patients, and patient representatives in hospital ethics committees – that could be engaged without extra training because they are already experienced in illustrating patient views (M. J. Armstrong et al., 2017). One study shared the perspective of a ‘professional’ patient representative, whose professional role provided additional insight and enabled her to share knowledge developed through her work (Biggane et al., 2022). This professional background and role enabled her to be involved throughout the clinical guideline development process, in a way that was different from other patient and public members (Biggane et al., 2022). Patient advocates, who had experience sharing their opinions in multidisciplinary groups and committees, felt more confident in guideline development groups (van Wersch & Eccles, 2001). The ‘helicopter view’ that certain patient representatives have is considered beneficial, meaning that they have an overview of a wide range of experiences from a range of service users (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Hämeen-Anttila et al., 2016; van der Ham et al., 2014). One study explicitly stated that guideline developers should aim to involve patients with prior engagement or guideline experience (Kim et al., 2021). In another study, patient/public members representing patient charities required less training in order to be involved due to their professional background and role (Biggane et al., 2022). Whilst these members did sometimes have difficulty understanding their role and the process of guideline development, when compared to patient members with lived experience of a condition, members from patient charities did not seem to struggle with the technical content of the guideline development. These “professional” patient/public members were often able to understand the scientific content and language they were presented with during the guideline development process (Biggane et al., 2022).

Guideline Developers Should Consider the Ratio and Diversity Characteristics of Patients and Physicians Within a Guideline Panel (CERQual: Moderate Confidence, n = 1 Study)

In one article (M. J. Armstrong et al., 2017), some patients felt that they would be comfortable as the only patient representative on a guideline panel, whereas others said this would make them feel like they didn’t belong and it would be intimidating. Those who expressed feeling more comfortable with the idea of serving as the sole patient representative were older and described more experience with either physicians or group dynamics. There was general consensus that including multiple patient representatives would be ideal. Some patients indicated that simply having multiple patient members would be enough for them to feel comfortable, while others indicated that they would want there to be an equal number of patients and physicians. Regardless of panel composition ratio, patients agreed that smaller panels would make them more comfortable presenting their views, particularly to physician panellists (M. J. Armstrong et al., 2017). In this study, most patients said that physician race or ethnicity would not matter, but several African-American participants indicated that they would be more comfortable if there was a physician of the same race on the panel.

Guideline Development Benefits From the Inclusion of Methodological Experts (CERQual: Moderate Confidence, n = 3 Studies)

Three studies identified the engagement of principal investigators as an enabler to developing guidelines (Chartres et al., 2020; Kim et al., 2021; Piggott et al., 2021). Specifically, they recommended including experts in qualitative research and systematic reviews to offset the training requirements needed by other guideline group members (Kim et al., 2021). In two studies, these “technical experts” or “methodologists” were described as working as part of an evidence review group, independent from the guideline development group (Chartres et al., 2020; Piggott et al., 2021). However, their inclusion in the larger guideline development group was identified as an important enabler for guideline development (Chartres et al., 2020; Kim et al., 2021; Piggott et al., 2021). Methodologists who are members of guideline development groups could facilitate discussions of the reviews presented by the evidence review groups. Their understanding of the challenges associated with evidence review could bridge a gap in communication between the guideline development group and evidence review groups (Chartres et al., 2020). By integrating the groups, with subgroups to evaluate the evidence for particular questions, the tensions from lack of contact and communication may be significantly reduced (Chartres et al., 2020).

Taking a Network Approach to Guideline Development Benefits the Engagement of Interest-Holders in the Guideline Development Process (CERQual: Low Confidence, n = 6 Studies)

Six studies suggested that a network approach (e.g. formation of a network of organisations or individuals in guideline development) could facilitate the engagement of patients, providers of care, payers of health services/research, policymakers and principal investigators (Carter et al., 1995; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004; Rapu, 2005; van der Ham et al., 2014). Guideline development groups can use these networks to identify potential members and/or to share learning, operational systems, and resources among organisations (Rapu, 2005).

Unclear Roles and Responsibilities for Guideline Development Group Members May Weaken or Disrupt Group Processes (CERQual: Moderate Confidence, n = 7 Studies)

When organisations do not have clear interest-holder engagement goals, the precise roles and responsibilities of each member of a guideline development group is often unclear. In the included studies, there was a lack of clear discussion about patients’, providers of care, and principal investigators’ roles and tasks in the guideline development groups, causing representatives to be uncertain as to what was expected of them (Carter et al., 1995; Jarret, 2004). Patients, in particular, were described as not being prepared for their role and were not aware of how their input influenced the guideline (Kim et al., 2021; Lanza & Ericsson, 2000). This can lead to members misinterpreting why they were selected and what they are supposed to do in the group, leading to tension between interest-holder groups (Chartres et al., 2020), creating confusion over how to use resources and which groups to involve (Carter et al., 1995; Jarret, 2004; van der Ham et al., 2014).

Failing to Manage Power Imbalances Between Patients and Providers of Care May Affect Group Dynamics and Limit Patient Engagement (CERQual: Moderate Confidence, n = 5 Studies)

Power differences between professionals and patients may cause issues within the guideline development group (M. J. Armstrong et al., 2017; Atkins et al., 2013; Carter et al., 1995; Harding et al., 2010; Kredo et al., 2018). If some personalities in the group are stronger than others, especially if members are posturing or power positioning, it may influence how patients participate in discussions. Guideline development meetings might be dominated by some interest-holders to the exclusion of others (Atkins et al., 2013). Perceived power imbalances can come from various sources - patients may not be comfortable if their own physician is on the panel (M. J. Armstrong et al., 2017), patients may interpret the use and explanation of jargon by other interest-holders as a display of power (Harding et al., 2010), and discussion chairs or leaders may not consider patient input. This can also extend to the national level, where territoriality and “turf protection” can interfere with national coordination and development efforts (Carter et al., 1995).

A Lack of Confidence in Their Skills and Legitimacy May Reduce Patient Members’ Level of Engagement (CERQual: Moderate Confidence, n = 5 Studies)

Lack of confidence was a commonly cited barrier to patients sharing their opinions in guideline development groups in five studies (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Harding et al., 2010; Kim et al., 2021), especially if they did not have prior experience participating in guideline development. Some patients did not want to appear uninformed or unhelpful when talking with experts and felt that their experiences were not as important as professional opinion (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Harding et al., 2010). This led to feelings of nervousness among patients and led to difficulties speaking up and voicing their opinions (Kim et al., 2021).

Clear Roles and Expectations for Interest-Holder Involvement in Guideline Development Facilitate Meaningful Engagement, Particularly for Patients and Principal Investigators (CERQual: Moderate Confidence, n = 12 Studies)

All interest-holders in a guideline development group should be aware of their roles, the tasks to be completed, and the timelines within which the guidelines are expected to be completed (Biggane et al., 2022; Boivin et al., 2010; Brouwers et al., 2017, 2018; Carter et al., 1995; Chalmers et al., 2017; Chartres et al., 2020; Den Breejen et al., 2014; Duff et al., 1996; Lanza & Ericsson, 2000; Piggott et al., 2021; Rapu, 2005). Among included studies, the most common role in need of more clarity was the role of patients. Patient roles should be clarified so that they understand what is expected of them, as well as the roles of others in the group. This can be done through orientation meetings delivered by a coordinator/navigator where patients can be told about and ask questions about the purpose and limits of their role (Brouwers et al., 2018). To clarify overall participant roles, guideline group members should have a strong understanding of their own and others’ experiences, clear project objectives and principles (Boivin et al., 2010; Brouwers et al., 2017, 2018; Carter et al., 1995; Chalmers et al., 2017; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Lanza & Ericsson, 2000; Rapu, 2005). One article highlighted that the role of the evidence review groups needs to be clearly articulated from the start and reinforced throughout the guideline development process (Chartres et al., 2020). The evidence review groups also need to feel appropriately supported in their roles when presenting the evidence reviews, particularly by the working groups chairs (Chartres et al., 2020).

Guideline Developers Should Foster Group Processes That Promote Patient Empowerment (CERQual: Low Confidence, n = 10 Studies)

Ten included studies discussed ways to empower patients (C. Armstrong et al., 2019; M. J. Armstrong & Bloom, 2017; Biggane et al., 2022; Brouwers et al., 2017; Chalmers et al., 2017; Daraz et al., 2019; Ilott et al., 2006; Jarret, 2004; Murad et al., 2019; Synnot et al., 2022; van Wersch & Eccles, 2001). Empowering and fostering self-determination in patients can facilitate their participation in guideline development and can promote the involvement of patients as equal members of guideline panels (Biggane et al., 2022; Murad et al., 2019). Personal motivators that empowered patient engagement included a feeling of appreciation from the members of the guideline development group (Brouwers et al., 2017), acknowledging patient effort (Synnot et al., 2022), providing information and reading material in advance (M. J. Armstrong et al., 2017; Synnot et al., 2022), and/or pre-specifying mechanisms for soliciting opinions (M. J. Armstrong et al., 2017). Panel chairs can also give the option of passing on questions or submitting written answers if patients feel uncomfortable in discussions, as well as creating separate spaces for patient feedback (M. J. Armstrong et al., 2017). Protected opportunities to share patient feedback, such as dedicated slots on meeting agendas, also helped patients to feel more capable of contributing (Jarret, 2004). Two studies reported that patient-specific training resulted in feelings of empowerment (Biggane et al., 2022) and improved self-efficacy and confidence (Daraz et al., 2019). During training, patients developed a script to use during panel deliberations (e.g., what to do when the conversation includes too much jargon) and an instruction sheet for other panellists on how to empower and engage patients (Daraz et al., 2019).

Allowing Interest-Holders to Identify Their Preferred Mode of Participation can Facilitate More Equitable and Accessible Engagement (CERQual: Low Confidence, n = 9 Studies)

There are often individual preferences around engagement, and guideline development interest-holders should be asked about these preferences (M. J. Armstrong et al., 2017; Synnot et al., 2022). Guideline developers can consider engaging all interest-holders in-person or virtually (C. Armstrong et al., 2019; Brouwers et al., 2018; Selby et al., 2017; Synnot et al., 2022). For providers of healthcare, there is a preference to network, collaborate and provide feedback in-person rather than online and to participate in structured discussions (Selby et al., 2017). Direct, in-person engagement can also encourage patients to participate. This may include holding structured workshops in-person whenever possible as well as prompting patients during discussions (Daraz et al., 2019; Selby et al., 2017). In-person meetings should be held at venues which are physically accessible. Where travel is prevented through ill health or other issues, teleconference facilities should be offered to avoid excluding patients’ valuable contributions (M. J. Armstrong et al., 2017; Chalmers et al., 2017; Selby et al., 2017). Patients and caregivers also reported online engagement processes to be a convenient way to overcome physical, geographical, and time constraints (C. Armstrong et al., 2019; Brouwers et al., 2018). However, online methods can preference the participation of well-educated, online-proficient consumers with internet access and this should be taken into consideration (Synnot et al., 2022). Online methods can be cost-effective ways to engage the public and patient groups (Bartelmann et al., 2019). In one study of the RAND/PPMD Patient-Centeredness Method (RPM), the convenience of an online approach was amplified by the asynchronous nature of the engagement process (C. Armstrong et al., 2019). Participants found the engagement process accommodating given that they could complete each round at their convenience rather than finishing all of it in one sitting (C. Armstrong et al., 2019). Online methods facilitate engagement for people with physical or social disabilities and people who are disadvantaged and living in low and low-middle income countries (Synnot et al., 2022). A study among people with disability (traumatic brain injury) found that focus group discussions and online Wiki’s were acceptable and feasible - with a slight preference for focus groups due to their social nature (Lamontagne et al., 2021). However, in another study use of a wiki platform by providers of care was minimal (Selby et al., 2017).

Involving a Skilled Moderator/Chair in the Guideline Development Process can Improve Group Processes and Facilitate Engagement (CERQual: High Confidence, n = 9 Studies)

Effective guideline group process is impacted by the quality of organisation and support from guideline sponsor organisations and effective chairing (Piggott et al., 2021). Chairs are essential to ensure that meetings occur according to an agenda and to provide clear expectations to ensure that discussions are held in a timely manner and activities are kept on target (Piggott et al., 2021). In order to give patients and caregivers more voice in guideline development groups, guideline groups should appoint a skilled moderator with “a great deal of sensitivity” to lead the group (M. J. Armstrong et al., 2017). Patient and caregiver members described a good chair as inclusive, skilled, open, honest, and able to influence guideline development group discussions (Jarret, 2004), and able to actively support involvement of diverse groups (Synnot et al., 2022). Two facilitation techniques were suggested for effectively engaging patients during meetings: notifying working group members of the training and resources that patients receive before meetings, and designating space for patient input in meeting agendas allowing patients to contextualise the clinical guideline content in a patient relevant manner (Biggane et al., 2022). An effective chair can also help to enforce accessible discussions by asking patients for their opinions, asking professionals to explain their terminology, and protecting participant boundaries (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Duff et al., 1996; Harding et al., 2010; Jarret, 2004; Kim et al., 2021). One study by Chartres et al. (2020) found that power imbalances between working groups [providers of care] and evidence review groups [principal investigators] could be minimised through appropriate facilitation. Facilitation can allow groups to feel their opinions are respected and appreciated throughout the guideline development process.

The Provision of Practical, Technical and Emotional Support Is a Facilitator of Engagement for all Interest-Holders (CERQual: Moderate Confidence, n = 9 Studies)

Support was identified as needed throughout the development process for all guideline development group members (Chartres et al., 2020; Den Breejen et al., 2014; Duff et al., 1996; Ilott et al., 2006; Jarret, 2004; Kim et al., 2021; Selby et al., 2017; Synnot et al., 2022; van der Ham et al., 2014). Guideline developers should designate an ongoing, proactive support person to provide practical, technical or emotional support (Synnot et al., 2022). Providers of healthcare described their employers as fully supportive of them doing guideline development activities during work hours, and support took the form of both time and encouragement (Ilott et al., 2006). Patients would benefit from support within their own patient organisations as well as collaborating centres (Jarret, 2004). This can be either process-related support, such as monitoring of patients and their needs throughout the process by the project manager, or content related support, for example by organising collective input from patient organisations. Guideline development group leaders should also consider contracting with local agencies or consultants to support engagement in certain geographic areas (Selby et al., 2017). Training all interest-holders in the skills required to take part in guideline development, such as communication and team-working, should also be provided (Kim et al., 2021).

Online Delphi Methods Facilitate Patient Engagement in Topic Selection and Priority Setting Exercises (CERQual: Low Confidence, n = 1 Study)

One article described how patients can help nominate and prioritise topics that they find important (C. Armstrong et al., 2019). Online modified-Delphi processes can be used by patients to systematically propose and develop consensus on the most important guideline topics. Specifically, patients could propose potential topics in an initial and open ended “brainstorming” round, and then prioritise them during subsequent rounds using close-ended rating or ranking questions. Alternatively, an online modified-Delphi process could help engage patients in prioritising an existing list of potential topics (C. Armstrong et al., 2019).

Patient Engagement May Be Limited by Concerns Around Confidentiality and Judgement in the Guideline Development Process (CERQual: Low Confidence, n = 3 Studies)

Some patients may refrain from sharing their opinion due to fears surrounding confidentiality or judgement (M. J. Armstrong et al., 2017). Patients may be worried about privacy issues, stigma, and/or denial when sharing personal experiences (Brouwers et al., 2017). Patients may be under the care of a member of the guideline development group, and this might not be a barrier for all patients, but it is possible that patients may feel coerced to participate or hide their true opinion if it is at odds with their physician’s opinion (Chalmers et al., 2017).

Guideline Developers Should Manage Conflicts of Interest and Patient Concerns Around Confidentiality (CERQual: Low Confidence, n = 4 Studies)

Project chairs should consider how they will manage patients’ conflicts of interest, especially if patients are receiving care from another guideline group member (Chalmers et al., 2017). While patients who had good relationships with their physicians felt comfortable sharing their opinions, this is not always the case (Chalmers et al., 2017). Chairs should take special care to maintain patient confidentiality and ensure that physicians do not disclose any personal patient information to the panel members without consent. Ideally, guideline developers should engage patients with no prior relationships with one another or the professionals on the panel (M. J. Armstrong et al., 2017). One article suggested that an online modified-Delphi process could be used by patients to indicate which conflicts of interest and mitigation strategies are most salient for developing a given guideline (C. Armstrong et al., 2019). In one study, guideline developers created working groups and ensured that a majority of members were without conflicts of interest. Members of the guideline development group without commercial conflicts were asked to adjudicate the final recommendations, thereby diluting the effects of any potential commercial bias (Selby et al., 2017).

Patients and the Public Perceived That Engagement in Health Outcome Selection Is Outside of Their Purview (CERQual: Low Confidence, n = 1 Study)

In one study conducted in the UK, patient and public members became involved in other aspects of the clinical guideline development, but their involvement in selecting health outcomes was relatively limited (Biggane et al., 2022). When asked, most patients and members of the public did not mention health outcomes as an area they were, or even wanted to be involved with. During committee meetings, they rarely participated in discussions about health outcomes or their selection and characterised this step of guideline development as a “system the technical team would go through”. How the role of patient/public members was perceived and the medical and scientific technicality of the guideline content all had a part in constraining their involvement in this topic (Biggane et al., 2022).

Time Constraints and Inflexible Timelines May Limit Patient and Public Engagement in Outcome Selection and Question Generation (CERQual: Low Confidence, n = 2 Studies)

Lack of time restricted the input of patient and public members in health outcome selection (Biggane et al., 2022) and patient and caregiver involvement in question generation (Köpke et al., 2018). One study used surveys to gather PICO questions from an international sample of patients and caregivers, and reported that this was time and resource consuming (Köpke et al., 2018). In the UK, this influenced guideline development later on, as the guideline development group was committed to the original questions and search terms developed earlier in the guideline process, and these were then not adaptable even if the resulting evidence was not relevant or understood by patient and public members (Biggane et al., 2022).

Patient and Public Engagement May Benefit From the Provision of Training Specific to Health Outcome Selection (CERQual: High Confidence, n = 1 Study)

The NICE Public Involvement Programme (PIP) facilitates training sessions to introduce patient and public members to the guideline development process, including topics such as understanding scientific evidence, how it is used, and what are outcomes (Biggane et al., 2022). Patient and public members largely described this training as “helpful” and “empowering”, praising the way their role, the process and the scientific terminology were explained (Biggane et al., 2022). It is important that clinical guideline developers continue to understand and embrace the value of providing training opportunities to the various members of the committee. Training should include the importance of patient and public member involvement and the rationale for outcome selection. This includes training for: the chair of the committee, health professionals, technical team members and patient/public members themselves. Opportunities for this could include expanding on the outcome section of the PIP training session or communicating it at committee level in the early meetings (Biggane et al., 2022).

Guideline Developers Should Consider Establishing Flexible Processes and Timelines (CERQual: High Confidence, n = 1 Study)

Guideline developers could explore whether there is flexibility in the timeline for the development of PICO questions and outcome selection. Specific suggestions include: seeking input to outcome selection at other junctures, establishing “break-out” working groups about specific topics, and increasing the timeframe for guideline development by having pre-meetings or engagements with committee members (Biggane et al., 2022).

Guideline Developers Should Use Empirical Methods for Engaging With Patients (CERQual: Low Confidence, n = 3 Studies)

Once the guideline development group is established, patients can assist with developing the PICO questions for the evidence review to ensure that its scope will be relevant and useful to real-world issues that the guideline aims to address. Online Delphi processes have been used to finalise the questions and outcomes (C. Armstrong et al., 2019). Specifically, patients can help refine clinical questions through an online modified-Delphi process that develops consensus on the importance of sub-populations, outcomes, aspects of treatments, and other elements (C. Armstrong et al., 2019). One study suggested that this information could also be collected qualitatively using focus group discussions (Biggane et al., 2022). They also recommended referring to already existing resources such as core outcome sets (Biggane et al., 2022). A third study used an international survey to select PICO questions, which allowed participation of a geographically varied population and by being anonymous facilitated openness and trustworthiness (Köpke et al., 2018). However, the authors noted that there was variable survey participation across countries, and that among those who did respond, they were online-proficient and well-educated participants, where the majority had completed post-secondary education (Köpke et al., 2018).

Guideline Development Should Involve ‘Champions’ and Representatives From Disease-specific Societies or Organisations (CERQual: High Confidence, n = 1 Study)

Disease-specific societies or advocacy groups can play a crucial role in the process of PICO question generation. In one study, the National Multiple Sclerosis (MS) Societies of participating European countries provided valuable feedback, particularly if interest-holders were both patients and MS Society members (Köpke et al., 2018). For example, one specific task force member was an MS patient and a member of the MS International Federation, and of the Danish national MS society. She was key in bridging the issues of the MS patients to those of the health professionals and researchers (Köpke et al., 2018), suggesting a valuable role of boundary spanners (i.e. institutions, groups, or individuals that straddle the divide between information producers and users) in health guideline development.

Patients’ Engagement May Be Limited by Feelings of Intimidation Derived From the Process or Other Group Members (CERQual: Moderate Confidence, n = 8 Studies)

Intimidation may be a barrier to engaging patients (M. J. Armstrong et al., 2017; Atkins et al., 2013; Brouwers et al., 2017, 2018; Duff et al., 1996; Harding et al., 2010; Jarret, 2004). In one study, a few participants said that preferences are hard to accurately capture, that clinicians did not always accept patient preferences, and it was challenging to meaningfully involve patients (Kim et al., 2021). Healthcare professional panel members are usually highly-trained and have experience working with other professionals but may not have experience working with those outside of their field and/or partnering with patients. Additionally, patient members may not have any training or experience within professional meetings, leading to hesitancy in commenting or questioning in the group discussions (M. J. Armstrong et al., 2017). This may be amplified if there is only one or few patients representative in the guideline development group (Atkins et al., 2013; Jarret, 2004). Some patients indicated that they did not want to appear uninformed or unhelpful, especially if the medical terminology and issues discussed are very complex (Brouwers et al., 2018). These participants may compare themselves to other guideline development group members or have less influence, which can undermine confidence.

Power Imbalances Between Patients and Providers of Care in Guideline Development Groups May Promote Unequal Influence in Decision-Making (CERQual: Moderate Confidence, n = 6 Studies)

There may be inequality in guideline development group members’ influence on decision-making (Atkins et al., 2013; Harding et al., 2010; Ilott et al., 2006; Jarret, 2004; Lanza & Ericsson, 2000; van de Bovenkamp & Zuiderent‐Jerak, 2013), especially if there is a perceived hierarchy of opinion where the perspectives of patient representatives are considered to be less valuable than those of healthcare providers (Atkins et al., 2013). Some guideline development group members may feel that patients do not easily fit into the decision-making structure (Harding et al., 2010). This can lead to limited influence of patient input, overturned group consensus, and a general undermining of the patient focus in guideline development (Harding et al., 2010). On the other hand, patients may not realise that their input is only one part of the guideline process and that the guideline development group is not obligated to follow the patients’ recommendations. One study reported that unequal influence was seen by some members as beneficial for decision making by preventing unfocussed discussion (Atkins et al., 2013).

Patient Engagement in Guideline Development May Be Limited by a Lack of Technical Skills Applicable to Reviewing Scientific Evidence (CERQual: High Confidence, n = 6 Studies)

Multiple studies mentioned the concern that the search, review, and discussion of scientific literature may be too technical or specialised for patient representatives (Hämeen-Anttila et al., 2016; Kredo et al., 2018; Schalkers et al., 2017; van de Bovenkamp & Zuiderent‐Jerak, 2013; van der Ham et al., 2014). The average patient’s lack of expertise - for both guideline development and literature reviews - was frequently cited as a barrier to engagement, for example: “He didn’t have the sort of skills – his contribution could only be […] from his experience that couldn’t meaningfully be fed into that process” (Brouwers et al., 2017).

The Impact and Depth of Patient Engagement May Be Limited by a Devaluation of Experiential Knowledge (CERQual: Moderate Confidence, n = 8 Studies)

The strong focus on evidence-based medicine and the systems used to evaluate scientific studies and categorise them in terms of the strength of the evidence makes it hard to give the experiential knowledge of patients a place in the guideline as it does not fit this categorization structure. One of the key challenges faced during guideline development processes is the dynamic tension between using an evidence-based approach (i.e., science, rigour and established expert contributions) versus tapping into tacit knowledge of all interest-holders (Walker et al., 2022). Guideline development group members may have different ways of defining evidence, ranging from rigorous scientific trials to “evidence of experience”. Within this, the medical knowledge of professionals is considered superior to the experiential knowledge of patients (Atkins et al., 2013; Harding et al., 2010; van de Bovenkamp & Zuiderent‐Jerak, 2013), and that this might be even more the case with children than with adults, suggesting that children’s experience of illness is excluded from guideline development (Schalkers et al., 2017). Several professionals indicated that it is often difficult to integrate input from patients into the guideline because it differed from other professional interest-holders (M. J. Armstrong et al., 2020) or focused on aspects of care, such as the organisation of care or the patient-provider relationship, which are often not the focus of a guideline (van der Ham et al., 2014). A single study also suggested that there are similar challenges engaging providers of healthcare and ensuring an appropriate balance between the evidence base, expert opinion and practitioner feedback (Selby et al., 2017).

Diverging Disciplines and Approaches Between ‘Evidence Review Groups’ (Technical Teams) and Guideline Development Groups May Promote Intergroup Conflict and Division (CERQual: Moderate Confidence, n = 1 Study)

Members of guideline development groups often have divergent disciplinary backgrounds (eg, physicians, toxicologists, clinical and public health researchers) compared to those on evidence review groups (eg, systematic review experts) and hold varying, sometimes conflicting beliefs about what constitutes ‘good’ evidence (Chartres et al., 2020). In one study, content experts often viewed ‘good’ or ‘important’ evidence from the perspective of their own knowledge and expertise and not from the standards of methodological rigour used by the evidence review groups (Chartres et al., 2020). These divergent roles and epistemological beliefs led to conflict and division between the groups. Many of the working committee members believed that in order to best protect the public’s health, the evidence presented by the review groups should at times be challenged (Chartres et al., 2020).

Guideline Developers Should Value Experiential Evidence and Elicit Patient Knowledge Beyond Guideline Development Group Membership (CERQual: Moderate Confidence, n = 4 Studies)

Patients are more likely to contribute if they feel appreciated by other members of the guideline development group and/or that their input matters. There are opportunities to access patient input from other sources over and above patient membership in the guideline development group (M. J. Armstrong et al., 2017; Hämeen-Anttila et al., 2016; Jarret, 2004). Patient and caregiver organisations have good channels for gathering information from the patient perspective. One possible means of gathering patient views were surveys distributed by the patient organisations (Hämeen-Anttila et al., 2016). Focus groups with patients and carers were also sources that guideline group members (including patient members) could draw on to inform their own contributions. Patients and caregivers appreciated interactive processes that include feedback loops (e.g. Delphi) and opportunities to directly engage with and learn from other patients (C. Armstrong et al., 2019). The iterative nature of data collection can foster better participant engagement by increasing their willingness to share and discuss their opinions and experiences with others, resulting in the development of new and shared perspectives (C. Armstrong et al., 2019).

Interest-Holder Engagement May Be Restricted by a Separation of Duties and Limited Communication Between Guideline Development Groups and Evidence Review Groups (CERQual: Moderate Confidence, n = 1 Study)

One study described how evidence review groups (composed primarily of researchers and methodologists) worked independently from the guideline development group for most of the process (Chartres et al., 2020). The transfer of ideas, knowledge and expertise pertaining to the inclusion and presentation of evidence was limited by this separation. By not working with the evidence review groups regularly from the outset and understanding the methods thoroughly, some guideline development group members felt limited in their ability to contribute to the guideline process due to their lack of methodological knowledge or disagreement with the included evidence. The irregularity of the meetings at which the evidence reviews groups presented their findings to the guideline development group meant it became an ineffective way of communicating how the evidence was being evaluated (Chartres et al., 2020). In turn, tensions developed between the two groups. Evidence review groups recognised an unequal and unfair power dynamic which led to them feeling that they were not valued or respected contributors to the guideline process (Chartres et al., 2020). The evidence review groups members felt that the tensions that were experienced could have been limited if guideline development groups were provided with clear information about the different roles of the two groups.

Interest-Holder Engagement in Evaluating Evidence May Be Limited by an Inadequate Understanding of Methods (CERQual: Low Confidence, n = 1 Study)

Guideline development group members and evidence review group members viewed understanding the methods [related to evaluating evidence] as a significant challenge to the current guideline development process (Chartres et al., 2020). Inadequate understanding of the methods restricts the level of input certain guideline development group members can have in the process and creates tensions between the groups (Chartres et al., 2020).

Providing Training on the Frameworks Employed in Assessing the Body of Evidence May Facilitate the Engagement of Guideline Group Members (CERQual: Moderate Confidence, n = 1 Study)

Pragmatic advice and training by the guideline development organisation for the guideline group members unfamiliar with the frameworks used to assess the body of evidence is necessary, not only at the commencement of the process but should be ongoing thereafter (Chartres et al., 2020).

Patient and Public Engagement May Be Limited by Their Lack of Content Knowledge, Clinical Expertise and Technical Skills (CERQual: Moderate Confidence, n = 12 Studies)

Some patients felt that they had a lack of clinical content knowledge, insufficient academic education, guideline process training, and/or computer skills, which would hinder the complex scientific discussion and the recommendation development process. Unless they are educated about the guideline topic or have specific prior knowledge and training, patients may stay quiet during group discussion (Harding et al., 2010; Schalkers et al., 2017; Selby et al., 2017; van de Bovenk & Zuiderent‐Jerak, 2013; van der Ham et al., 2014). Additional barriers included the need for in-depth scientific discussion, which may not always be accessible for patients or caregivers, leading to the concern that discussions may be stifled if conversations are constantly being interrupted for lay explanations (M. J. Armstrong et al., 2020; M. J. Armstrong & Bloom, 2017; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014). In one study, discussions among health professionals in committee meetings for this guideline indicated that they also believed that the content was too clinically driven for lay members to understand (Biggane et al., 2022).

Insufficient and Competing Demands of Time May Limit Interest-Holder Engagement in Recommendation Development (CERQual: Moderate Confidence, n = 11 Studies)

Time was a frequently cited barrier across multiple studies regarding the engagement of patients, caregivers, patient organisations, and providers of care (M. J. Armstrong & Bloom, 2017; Atkins et al., 2013; Bartelmann et al., 2019; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Kredo et al., 2018). This included insufficient time for discussion and recommendation formulation, as well as the time-consuming nature of travel and attending meetings. Participants may feel rushed or unable to fulfil their responsibilities. For example, challenges were encountered in getting responses from professional societies within the four-week consultation deadlines, and these problems were more severe for organisations with limited capacity (under 2,000 members) (Ilott et al., 2006). One study noted that it was difficult to keep patients interested in attending meetings because of conflicts with patients’ work or personal lives (Brouwers et al., 2018). A lack of dedicated time for guideline development may also lead to guideline development group members working voluntarily on their own time and afterhours (M. J. Armstrong et al., 2017; Atkins et al., 2013; Brouwers et al., 2017, 2018; Chalmers et al., 2017; Den Breejen et al., 2014; Ilott et al., 2006; Kredo et al., 2018). The length of time required to develop guidelines may also prevent healthcare professionals from engaging in their development (Bartelmann et al., 2019). Clear timelines are also needed for the technical experts conducting evidence syntheses, so that sufficient time is allowed for the review of evidence in advance of guideline meetings (Piggott et al., 2021). Patients should be given sufficient notice of meetings and allocated enough time to understand the materials and provide feedback (Brouwers et al., 2018; Chalmers et al., 2017; Duff et al., 1996).

Unequal Access to Information Technology Resources May Limit Interest-Holders’ Ability to Engage in Guideline Development (CERQual: Moderate Confidence, n = 3 Studies)

Two studies, conducted among patients (Synnot et al., 2022) and providers of care (Kredo et al., 2018) identified that the guideline development process favoured those with access to and the ability to use the Internet and technology. One study noted that a patient representative had trouble carrying out her work because she had no printer and the documents she needed were soft copy only (Jarret, 2004).

Technical Concepts and Professional Jargon May Negatively Impact Patient and Public Comfortability and Ability to Contribute to the Guideline Development Process (CERQual: Moderate Confidence, n = 6 Studies)

Patients thought meeting dialogue was sometimes overly technical and that professionals’ use of jargon was like “speaking a foreign language” and represents a display of power (M. J. Armstrong et al., 2020; Biggane et al., 2022). This technical content comprised terminologies, abbreviations and topic content which lay members believed were largely only accessible to health professionals or those with specialised knowledge (Biggane et al., 2022). Many patients may not be familiar with the medical language spoken and ask for clarification of terms and abbreviations (M. J. Armstrong et al., 2020). Patients without some research knowledge reported difficulties grasping the concept of the evidence framework and the statistics surrounding it. Some patients who found the documents and reading material too technical or scientific decided that they had no contribution to make on these issues and felt excluded (Harding et al., 2010; Jarret, 2004; van Wersch & Eccles, 2001). Such patients were reported to be “participating observers of technical language to which they could hardly offer any input” (Brouwers et al., 2018).

Guideline Developers Should Provide Advance Preparation Materials and Resources Prior to Meetings and Throughout the Project (CERQual: Moderate Confidence, n = 5 Studies)

Providing resources to all interest-holders, especially hard-copy documents, before meetings and offering opportunities for clarification can help all guideline development group members contribute effectively (Jarret, 2004). This may include supplying meeting agendas and talking points, questions, guidance and terms of reference documents, and/or materials to take notes with (Brouwers et al., 2018). There should also be a point of contact for guideline development groups members (M. J. Armstrong et al., 2017); those in charge should make time to return calls and keep track of their contacts. It is particularly helpful for patients to meet with the chairs prior to meetings (and particularly the first meeting) so that they know what to expect and get to know some other people in the meeting prior to joining larger group meetings (M. J. Armstrong et al., 2017; Brouwers et al., 2018; Chalmers et al., 2017; Jarret, 2004; Lanza & Ericsson, 2000).

Guideline Developers Should Offer Explanations and Promote the Use of Plain Language to Ensure That Participation in the Guideline Development Process Is Accessible to all Members (CERQual: Moderate Confidence, n = 4 Studies)

Guideline developers should ensure the language used in all aspects of the guideline development is accessible to all committee members (Biggane et al., 2022). In particular technical experts must communicate concisely, effectively, and collaboratively toward the production of the guidelines (Piggott et al., 2021), preferably in plain language avoiding scientific or medical abbreviations (Biggane et al., 2022). Other activities may include translation of scientific literature and content into plain language formats as needed (Biggane et al., 2022). A pivotal success factor for engaging patients seemed to be the extent to which people explained terms to one another. Project chairs or navigators can meet with patients prior to meetings so that patients can have an explanation of the context of the expected discussion. Guideline development group members can also use natural breaks in discussion to explain to patients in lay terms what is being discussed and ask patients specific questions where their input is needed. Some people linked this with an (often subtle) display of power which could be redistributed. Others felt these strategies contributed by preventing unhelpful assumptions from influencing proceedings (Chalmers et al., 2017; Harding et al., 2010).

Guideline Developers Should Consider Using Online Methods/Tools to Refine Recommendations (CERQual: Low Confidence, n = 3 Studies)

We identified 3 studies which suggested the use of online tools to engage interest-holders in the refinement or wording of recommendations (C. Armstrong et al., 2019; Lamontagne et al., 2021; Pottie et al., 2021). The GRADE-FACE is proposed as an efficient means to collect interest-holder perspectives on drafted recommendations whereby respondents assess the feasibility, acceptability, cost, and equity of each drafted recommendation on a GRADE rating scale and provide qualitative comments to justify and explain their response (Pottie et al., 2021). The authors note that for underserved and disadvantaged populations, electronic surveys may not be feasible or appropriate. In-person semistructured interviews may be best suited to collect guideline perspectives from these groups (Pottie et al., 2021). Similarly, the RPM is an online modified-Delphi approach to patient engagement in guideline development that allows patients and their representatives to provide input on the patient-centeredness of draft guideline recommendations by participating in an idea generational round, two rating rounds, and an online discussion round (C. Armstrong et al., 2019). The RPM can also be used to explore the existence of consensus among key interest-holders on other dimensions of the Evidence-to-Decision framework, such as feasibility, equity, resource use and issues related to disadvantaged patient subpopulations (C. Armstrong et al., 2019). A third study proposed an online wiki approach (Lamontagne et al., 2021). Participants considered the wiki method effective to the extent that it allowed for analysis and reflection before answering and because participants could express themselves without being stopped or interrupted. Other key advantages mentioned included anonymity, not feeling judged by others, the freedom to choose the time and place to answer the wiki questions, and doing it at the participant’s own pace, respecting his or her capacity. However, participants were individuals with traumatic brain injury, and generally expressed preference for in-person options such as focus groups, as they more easily fit with the communication habits of individuals with a head injury who prefer oral expression to writing (Lamontagne et al., 2021).

Interest-Holder Engagement in the External Review Process May Benefit From Longer Review Periods Combined With Individualised Reminder Emails (CERQual: Low Confidence, n = 1 Study)

We identified only one study which reported facilitators to interest-holder engagement during the external peer review process of a guideline (Bartelmann et al., 2019). They reported measures to encourage people to take part in the external review of guidelines might include longer review periods combined with individualised reminder emails to people within all categories of interest-holders. Paper-based advertisements in clinical practices and online advertising on patient-focused websites might be an option to identify patients and members of the general public who could participate. These measures, however, require time and funding that may not be available to the average guideline development group. Less expensive solutions might include the use of online platforms that allow external reviewers of guidelines to obtain recognition for their efforts even in cases where their reviews are anonymous. Having external review efforts formally recognised and rewarded by medical faculties or colleges might also have a salutary effect on participation rates (Bartelmann et al., 2019).

Online Delphi Methods can Be Used to Engage Patients in Identifying Implementation Considerations and Dissemination Strategies (CERQual: Low Confidence, n = 1 Study)

Patients can assist in disseminating and implementing guideline recommendations. For example, patients and patient advocacy groups can endorse guidelines to improve their legitimacy and trustworthiness to patient communities, as well as increasing patient adherence to guideline recommendations. Online modified-Delphi processes can be used to engage patients in identifying items to include in decision aids, as well as barriers and solutions to guideline uptake (C. Armstrong et al., 2019).

Guideline Developers May Face Challenges Engaging Interest-Holders in Guideline Updates due to Insufficient Financial Resources (CERQual: Low Confidence, n = 2 Studies)

Two studies reported barriers to maintaining interest-holder engagement for patients, providers of care, policy makers, and principal investigators during guideline updates (Selby et al., 2017; van der Ham et al., 2014). In one study, lack of ongoing funding prevented the secretariat from keeping the original guideline development group engaged in guideline activities (Selby et al., 2017). van der Ham et al. (2014) also suggests that it can be difficult for patients to ‘catch up’ and meaningfully contribute if they were not involved in the original guideline.

Using an Online Delphi Process to Assess the Patient-Centeredness of Guideline Recommendations May Help Facilitate Patient Engagement in Guideline Updates (CERQual: Low Confidence, n = 1 Study)

According to one article, patients can assist in updating and evaluating guidelines, such as determining if and when guidelines require an update, assessing whether patients made meaningful contributions to a guideline, and informing improvements to future patient engagement strategies in guideline development. To facilitate patient engagement at this stage, online modified-Delphi processes can be used to continually assess the current patient centeredness of guideline recommendations. For example, patients can define and prioritise significant changes in views from the patient community since a guideline has been developed, such as outcomes and interventions that should be included in a guideline update (C. Armstrong et al., 2019).