Abstract
Systematic reviews of qualitative research (‘qualitative evidence syntheses’) are increasingly popular and represent a potentially important source of information about people’s views, needs and experiences. Since 2013, Cochrane has published qualitative evidence syntheses, and the Cochrane Effective Practice and Organisation of Care group has been involved in the majority of these reviews. But more guidance is needed on how to prepare these reviews in an environment that is more familiar with reviews of quantitative research. In this paper, we describe and reflect on how Cochrane qualitative evidence syntheses differ from reviews of intervention effectiveness and how these differences have influenced the guidance developed by the EPOC group. In particular, we discuss how it has been important to display to end users, firstly, that qualitative evidence syntheses are carried out with rigour and transparency, and secondly, that these quality standards need to reflect qualitative research traditions. We also discuss lessons that reviews of effectiveness might learn from qualitative research.
Keywords
Introduction
Cochrane has existed for over 25 years and is considered a leader in the preparation of systematic reviews of the effectiveness of healthcare interventions. Over the past few years, Cochrane has also started to publish other types of reviews, and in 2013, Cochrane published its first systematic review of qualitative research (or ‘qualitative evidence synthesis’) (Glenton et al., 2013). By September 2021, The Cochrane Library had published 32 qualitative evidence syntheses and synthesis protocols. Most of these reviews address questions related to the views and experiences of patients and the public, healthcare workers and other stakeholders of healthcare interventions, their acceptability, feasibility and implementation. In addition, Cochrane has published a number of mixed methods reviews that address these types of questions alongside questions of intervention effectiveness (e.g. (Hurley et al., 2018; Vasudevan et al., 2018).
While the number of qualitative evidence syntheses increase each year, these still make up a small proportion of the over 8000 Cochrane Reviews in the Cochrane Library. Cochrane’s methodological guidance, editorial support and publication processes are still very much focused on reviews of intervention effectiveness, and the skills and competencies present within most Cochrane Review Groups reflect this focus. Cochrane Review Groups may therefore struggle to support the preparation of qualitative evidence syntheses and many Review Groups have not extended their work to include this type of review. But the growing popularity of qualitative evidence syntheses indicates a need for more guidance.
Guidance on how to prepare and report qualitative evidence syntheses has increased rapidly over the past few years, due to the efforts of Cochrane’s Qualitative and Implementation Methods Group (J. Noyes, Booth, Cargo, et al., 2018; J. Noyes et al., 2019) and initiatives outside of Cochrane, such as the EMERGE reporting guidance for meta-ethnographies (France et al., 2019). However, as noted in an accompanying paper (Glenton et al., 2021), guidance is still lacking for a number of topics, including the use of sampling approaches, dealing with studies published in multiple languages, addressing review author reflexivity and conflict of interest issues, developing implications for practice and for future research and preparing abstracts and plain language summaries. Where guidance does exist, our review authors sometimes find it difficult to interpret or apply it in practice, often because the methods are difficult to operationalise (Tricco et al., 2016) or because some review authors are relatively new to the field of qualitative evidence synthesis.
Cochrane has 51 Review Groups, each focusing on a specific topic area, and each with its own editorial team that is responsible for managing and publishing reviews within this topic area. Many Cochrane Reviews are commissioned by decision makers such as governments and international agencies. The reviews are then prepared by review authors from universities and organisations around the world, with support from the editorial team. Cochrane Reviews are generally considered to be of high quality, in part because they are expected to meet specific quality standards for conduct and reporting. Cochrane’s Effective Practice and Organisation of Care Group (EPOC) is one of the Cochrane Review Groups that is most active in supporting the production of qualitative evidence syntheses. This is partly because the editorial team already included people with relevant methodological experience and is also a consequence of requests for this type of evidence from review commissioners such as the World Health Organization (Glenton et al., 2016).
To further guide EPOC’s editorial team and its review authors and to support consistent standards of conduct and reporting, EPOC recently developed a template and supporting guidance for their qualitative evidence syntheses (Glenton, Bohren, et al., 2020). As an editorial team working within an organisation that mainly focuses on effectiveness reviews, this also provided us with the opportunity to discuss where, how and why qualitative evidence syntheses differ from reviews of effectiveness.
Objective
The objective of this paper is to describe and reflect on how Cochrane qualitative evidence syntheses differ from reviews of intervention effectiveness, and how these differences have influenced our guidance for Cochrane EPOC qualitative evidence syntheses.
Materials and Methods
In 2019, members of Cochrane EPOC’s editorial team developed a template for EPOC qualitative evidence syntheses (Glenton, Bohren, et al., 2020). By ‘template’ we mean a structure for writing a protocol or full qualitative evidence synthesis. Our template includes suggested subheadings and standardised text for key sections of a protocol and review; short explanations regarding key content for a protocol and review; examples; and links to further resources. In other parallel processes, we also developed supporting guidance that focused specifically on sampling approaches and on preparing plain language summaries in the context of qualitative evidence syntheses approaches (Ames et al., 2019; EPOC, 2019; Glenton, Nilsen, & Fønhus, 2020).
The template and guidance were further developed and refined through discussions and feedback from EPOC’s information specialist and from Cochrane review authors working on qualitative evidence syntheses. Throughout the preparation of each review, the editorial team and review authors discussed challenges and explored solutions on a regular basis through email or teleconference. Most of the review teams also had 2- to 3-day face-to-face meetings with the EPOC editorial team, and the editorial team kept notes of emerging challenges and solutions. We describe the process by which the template and guidance were developed in more detail in another paper (Glenton et al., 2021).
Throughout the development of the template and guidance, the editorial team and review authors discussed how qualitative evidence syntheses differed from reviews of effectiveness and how we could find solutions that followed the principles of qualitative research while meeting Cochrane reporting standards. The editorial team also kept notes of these discussions and of the solutions we identified. During these discussions, we turned to other sources of guidance for qualitative evidence syntheses, including guidance developed by the Cochrane Qualitative and Implementation Methods Group and the EMERGE reporting guidance for meta-ethnographies (France et al., 2019). In this paper, we focus on topics that have received less attention in other existing guidance.
Discussion
In the following, we discuss several areas where qualitative evidence syntheses differ from reviews of effectiveness and we describe how the guidance we have developed reflects these differences. This includes issues tied to the inclusion of studies in multiple languages, study sampling approaches, review author reflexivity, development of implications for practice and implications for future research, potential conflicts of interest, and the preparation of abstracts and plain language summaries. We start by discussing an overarching issue: how the epistemological perspectives that underpin qualitative research differ from the perspectives reflected in quantitative research traditions more broadly and reviews of effectiveness specifically, and how this is reflected in the methodological responsiveness and flexibility of qualitative evidence syntheses.
Acknowledging the need for methodological responsiveness and flexibility
Authors of Cochrane effectiveness reviews are required to publish a protocol where they describe the methods they plan to use. By requiring this from review authors, Cochrane aims to reduce the impact of review author bias and promote transparency (Lasserson et al., 2019). Review authors can make changes after their work has begun, and frequently do so. However, this is not encouraged and changes that are made after the protocol is published need to be documented and justified in the final review. Within qualitative inquiry, on the other hand, the researcher is actively encouraged to respond to the data throughout the research journey. While qualitative researchers also start with a research question, they expect this question or sub-questions to evolve, which again is likely to have implications for data collection and analysis (Agee, 2009).
In our guidance, we acknowledge this qualitative research tradition by highlighting areas where changes from the protocol to the review are likely. In particular, we point out that the review authors’ planned sampling and data analysis approach may change once they have familiarised themselves with the data available to them. Similarly, where they plan to link their review findings to the results of related effectiveness reviews – a practice promoted within Cochrane (J. Noyes et al., 2019) – we indicate that they are not expected to make a decision regarding how to do this until after they have developed their own review findings. Methodological flexibility does not imply, however, that reporting should be any less rigorous than for reviews of effectiveness. As is the case for effectiveness research, qualitative research traditions encourage clear and auditable reporting to demonstrate dependability and confirmability and thereby trustworthiness (Lincoln & Guba, 1986). We therefore specify the type of information we expect the review authors to report and offer examples of text from other reviews and protocols. And as for all Cochrane Reviews (Page et al., 2019), we require that review authors describe any changes from the protocol to the review and provide the rationale for the changes.
Dealing with studies in multiple languages
Cochrane encourages authors of effectiveness reviews to search for and include studies published in any language (Higgins et al., 2019). This is driven by concerns about language bias, particularly the concern that some types of results may be excluded if we limit our reviews to (typically) English-language studies, although the research on this topic is inconsistent (Lefebvre et al., 2019).
There is no reason to believe that the potential for language bias is any less for qualitative evidence syntheses, and issues related to language of publication may also contribute to dissemination bias in qualitative evidence syntheses (Toews et al., 2016, 2017). However, the practical implications of including studies in any language are likely to be more serious. First, because of the textual nature of qualitative findings, the volume of data that review authors need to translate is likely to be larger in a qualitative study than in a quantitative study. In addition, a good translation of a qualitative study is likely to require an understanding of study context. This is not to say that study context does not matter for quantitative studies. For instance, descriptions of the intervention and outcomes can also be context-specific and open to interpretation. However, much of the data in these studies is numerical, and translation is therefore more straightforward. In qualitative research, on the other hand, data may have been gathered in one language, translated to another language for publication, and then translated again for use in the review. Translation of qualitative studies involves interpreting and precisely portraying the contextual and cultural meaning of textual data such as study authors’ interpretations and participant quotations that may include idioms and metaphors in addition to descriptions of the context, data collection and analysis methods. Capturing these nuances is challenging, and requires resources, skills, an understanding of the cultural and linguistic context and reflexive engagement with the translation process and outputs (Helmich et al., 2017; van Nes et al., 2010).
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C, Bohren MA, Downe S, Paulsen EJ, & S, 2019).
Using sampling approaches
Authors of effectiveness reviews aim to identify and analyse all studies that meet the eligibility criteria. By identifying as many relevant studies as possible, review authors aim to avoid selection bias and achieve a reliable estimate of effects (Lefebvre et al., 2019). While the Cochrane Handbook acknowledges that review authors may need to adapt to time or resource constraints, the Handbook also underlines that the aim is to identify and include as many relevant studies as possible (Lefebvre et al., 2019).
In contrast, qualitative evidence synthesis aims for variations in concepts rather than including all eligible studies. Study quantity and data richness is not unimportant in a qualitative evidence synthesis, and small numbers of studies or participants and thin data can threaten our ability to make broad claims and can lead us to downgrade our confidence in a review finding (Glenton et al., 2018). However, as review authors are expected to familiarise and immerse themselves in the study data and context, large numbers of studies can also prevent us from carrying out a thorough analysis that explores and explains a phenomenon comprehensively (Glenton et al., 2018). In syntheses where many studies meet the eligibility criteria, one approach is therefore to purposively sample a selection of these studies (Suri, 2011).
Within primary qualitative research, purposive sampling (i.e. selecting cases that are likely to be information-rich in relation to our research question) is a familiar approach (Miles & Huberman, 1994; Patton, 2002). Several strategies have been suggested for how to apply this approach in the context of qualitative evidence synthesis (J. Noyes, Booth, Flemming, et al., 2018; Suri, 2011). We have referred to some of these strategies in the EPOC template and have also prepared additional guidance on how to develop a purposive sampling frame in the context of a qualitative evidence synthesis (EPOC, 2019). In some of our reviews, review authors have used a sampling frame approach to sample studies based on their relevance to the review topic or their data richness (Ames et al., 2019). Others have used this approach to sample studies based on their geographical setting, type of participant group or type of condition in order to achieve maximum variation (S. Downe et al., 2019; Houghton et al., 2020; Karimi-Shahanjarini et al., 2019). Theory-based sampling and sampling based on confirming and disconfirming cases (Suri, 2011) are approaches that may also be helpful where a synthesis aims to explore theoretical constructs in relation to the review question. These approaches have not yet been sufficiently explored in EPOC qualitative evidence syntheses, and a number of questions remain about applying these approaches within such syntheses (Dixon-Woods et al., 2006).
So far, in the syntheses in which we have been involved, we have only applied a sampling approach before carrying out our data synthesis. We would also like to explore further whether it makes sense to sample studies more iteratively. For instance, we could return to eligible but un-sampled studies to search for additional data after we have synthesised the data from the initial group of studies and assessed our confidence in the draft findings. In Cochrane qualitative evidence syntheses, we use the GRADE-CERQual approach to assess our confidence in each review finding (Lewin et al., 2018). Where we have downgraded our assessment because of concerns about data adequacy (one of GRADE-CERQual’s four components (Glenton et al., 2018), it may make sense to go to the studies that were eligible but not sampled to see if this lack of data can be addressed. Similarly, where we have downgraded draft findings because of concerns about the coherence of the data (Colvin et al., 2018), the relevance of the studies (J. Noyes, Booth, Lewin, et al., 2018), or their methodological limitations (Munthe-Kaas et al., 2018), we may want to go back to the wider group of eligible studies.
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C et al., 2019).
Another topic of discussion, particularly in an environment more familiar with effectiveness research, is that sampling may lead review authors to give preference to studies that are in alignment with their world views, which could be described as a form of ‘groupthink’ (Schippers & Rus, 2021). While purposive sampling is an accepted approach in qualitative research traditions, clear reporting, transparency and reflexivity about these processes is therefore important, a topic we discuss further below.
Encouraging review author reflexivity
Many of the methodological recommendations made for Cochrane Reviews of effectiveness are there to protect the data from the personal biases of the review authors. This reflects a view of evidence as something that can be achieved in its ‘pure state’, once all sources of bias have been removed, including the perspective of the review author (a standpoint sometimes described with reference to the phrase ‘a view from nowhere’ (Nagel, 1986)). However, most qualitative research belongs to an epistemological tradition that sees evidence as a co-production between researchers and their informants (or, in our case, between the review authors and their included studies). Here, data is always a ‘view from somewhere’ (Harraway, 1988), and the data are seen as inextricably linked with the perspective of the researcher. But while the influence of the researcher is seen as unavoidable, qualitative researchers are encouraged to consider and describe how they and their research have interacted with and influenced each other, a concept referred to as ‘researcher reflexivity’ (Edge, 2011).
Despite this ideal, the primary qualitative studies included in our reviews typically show a lack of such discussions or descriptions of reflexivity. This problem has also been noted by others (Newton et al., 2012). In Cochrane’s first qualitative evidence (Glenton et al., 2013), we as review authors reported this problem with the included studies. But, ironically, we also failed to practice any type of researcher reflexivity ourselves. EPOC’s review authors first started applying the concept of review author reflexivity within our protocols in 2016 (S. Downe et al., 2016).
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C et al., 2019).
Developing implications for practice
The ‘Implications for practice’ section is mandatory for all Cochrane Reviews, and a reminder of Cochrane’s primary goal – to support people in making decisions about healthcare (Cochrane, 2019a). However, review authors need to balance this with Cochrane’s policy of not making recommendations. This policy is an acknowledgement that people’s decisions should be based on a wide array of evidence and also depend on their settings and individual circumstances (Page et al., 2019).
When authors of effectiveness reviews prepare this section, EPOC suggests that review authors focus on helping the end user think through a range of factors in addition to the effects of the intervention, including the intervention’s cost, equity implications, acceptability and feasibility (EPOC, 2017). This can be difficult to do well and requires authors of effectiveness reviews to go beyond their own review results and to consider aspects such as the management and organisation of care that they may not have knowledge or experience of. Qualitative evidence syntheses, on the other hand, can be particularly useful sources of information when it comes to several of these factors, and the implications for practice section can therefore be more straightforward to prepare without having to go beyond the findings of the review (Glenton et al., 2019).
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C et al., 2019).
These sections have been shown to be important sources of information when developing implementation considerations for WHO guideline recommendations, and we have recently described the use of qualitative evidence syntheses for this purpose in more detail (Glenton et al., 2019). In addition, authors of intervention effectiveness reviews linked to the topic of the qualitative evidence synthesis could also use this information in their own Implications for practice section, and this is already being done in our mixed methods reviews, which include both effectiveness and qualitative studies (Agarwal et al., 2020; Vasudevan et al., 2021). Further exploration is needed of how different stakeholders in different contexts use these prompts, including the extent to which they are able to bring them together with local experience and evidence to inform decisions, and whether applicability checklists may be useful in this process (Booth et al., 2019; Lewin et al., 2009).
Developing implications for future research
In addition to supporting decision makers, Cochrane Reviews aim to provide researchers and research funders with an overview of any gaps or limitations in the existing research. As Brown et al. point out, ‘“More research is needed” is a conclusion that fits most systematic reviews. But authors need to be more specific about what exactly is required’ (Brown et al., 2006).
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C et al., 2019).
We also suggest that review authors use their GRADE-CERQual assessments to identify gaps in and limitations of the existing research. For instance, where review authors’ GRADE-CERQual assessments of confidence in the review findings commonly point to concerns about data adequacy, coherence or relevance, this may suggest a need for more data around certain topics or from specific population groups or settings. Similarly, the review authors’ assessments of the studies’ methodological limitations can help identify room for improvement in the design or conduct of future studies.
In reviews of effectiveness, review authors commonly identify gaps in the evidence by pointing to pre-defined outcomes for which eligible studies were not found. While authors of qualitative evidence syntheses do not pre-define outcomes, their reviews often aim to test or develop models, frameworks or hypotheses (Harris et al., 2018). In these cases, we encourage review authors to identify types of data that could have helped them test or build these models.
Authors of effectiveness reviews also consider whether some types of research findings may be systematically under-represented because of the nature and direction of the results (Sterne et al., 2011). So-called ‘publication bias’ or ‘dissemination bias’ is one of the GRADE components when assessing our confidence in findings from effectiveness reviews (Schunemann et al., 2013). However, this component is not currently included in the GRADE-CERQual approach as we have large gaps in our understanding of whether and how dissemination bias might impact on the findings of qualitative evidence syntheses and on assessments of confidence in these findings (Booth et al., 2018). The GRADE-CERQual group has initiated research activities that we hope will move forward this field (Toews et al., 2016, 2017).
Declaring potential conflicts of interest
As previously mentioned, many of Cochrane’s standards are designed to safeguard reviews from the personal biases of the review authors. In addition, all Cochrane Reviews are expected to include a declaration of interest statement from each review author (Cochrane, 2014a). Cochrane’s editorial and publishing policies also indicate when a person’s declaration of interest statement is serious enough to prohibit him or her from participating in the review team (Cochrane, 2014b).
These statements, and decisions to exclude potential authors from a review team, have focused primarily on financial interests and ties. But there is a growing interest within Cochrane in exploring non-financial conflicts of interest, including academic, professional and personal interests. This new focus is reflected in the organisation’s work to update its conflict of interest policies (Cochrane, 2019b).
One challenge with this new approach is that it is far more difficult to introduce clear cut-offs for review authorship when it comes to non-financial conflicts of interest. Cochrane acknowledges that this type of conflict ‘is impossible to avoid and affects everybody – even those who believe themselves to be immune for any type of outside influence’ (Cochrane, 2019b). Cochrane is now discussing the type of non-financial conflicts of interests that should be included in a Cochrane policy, and whether these conflicts should simply be declared or whether they should prevent authorship (Cochrane, 2019b).
We would argue that there are alternative approaches to authorship exclusion or declaration. The concept of reflexivity represents a possible third solution for all review authors, including authors of effectiveness reviews, whereby review authors are not only encouraged to ‘declare’ their non-financial ‘interests’, but are also expected to discuss and respond to them actively throughout the review process.
Excerpt from EPOC’s qualitative evidence synthesis template (Glenton C et al., 2019).
Preparing abstracts and plain language summaries
EPOC staff have for several years been involved in research and development around summarising reviews of effectiveness, for instance, in plain language summaries, abstracts and review summaries for policy makers (Glenton, Nilsen, Fønhus, Goudie, & Noonan, 2020; Glenton et al., 2010; Glenton et al., 2006; Rosenbaum, Glenton, Nylund, & Oxman, 2010; Rosenbaum, Glenton, & Oxman, 2010; Rosenbaum et al., 2011). An overarching aim of this work has been to ensure that summaries give a brief and accessible but also reasonably precise summary of a review’s most important findings. EPOC recommends that authors of effectiveness reviews present the most important outcomes in the review abstract and plain language summary, including outcomes where no studies were found, as well as the GRADE assessment of our confidence in this evidence (EPOC, 2018). EPOC suggests that review authors use the Summary of Findings table as the basis for these summaries as this table contains the most important outcomes as well as the GRADE assessment. EPOC also encourages review authors to use standardised plain language statements that reflect both the magnitude of effect and our confidence in the evidence (Glenton, Nilsen, Fønhus, et al., 2020; Glenton et al., 2010).
Excerpt from EPOC’s Plain Language Summary template (Glenton, Nilsen, & Fønhus, 2020).
Lessons for reviews of effectiveness
As Booth argues (Booth, 2019), qualitative evidence syntheses can be said to have a ‘dual heritage’ in the sense that they draw on methods from both qualitative primary research and systematic reviews of effectiveness. Authors of Cochrane qualitative evidence syntheses also need to find solutions that ‘satisfy the rigor required by review methods, coupled with sensitivity to the qualitative paradigm’ (Booth, 2019). In this paper, we have focused on the differences between qualitative evidence syntheses and reviews of effectiveness, and how the design and conduct of these two types of reviews reflect different methodological and philosophical traditions. The EPOC template for qualitative evidence syntheses has attempted to reflect these differences and to clarify how qualitative evidence syntheses can and should differ from reviews of effectiveness. However, we would argue that qualitative evidence syntheses, and the qualitative traditions and methods on which these draw, also provide learning for reviews of intervention effectiveness.
One such aspect is the acknowledgement that effectiveness reviews inevitably evolve between the protocol and review stages. It is not uncommon for review authors to continue to refine their scope and methods after the protocol has been published. For instance, early searches may identify studies that challenge the review authors’ understanding of a topic and the inclusion criteria they have agreed upon with regard to populations, interventions, outcomes or comparisons (Lefebvre et al., 2019). Reviews that deal with particularly complex topics and questions, or that focus on a particular outcome rather than an intervention, may be especially challenging (Lorenc et al., 2016; Petticrew et al., 2013, 2019; Shepperd et al., 2009) and changes to inclusion criteria are likely to occur. Further discussion is needed regarding where the line should go between an acceptable refinement of review methods and ‘cherry picking’ and ‘review author bias’.
Another area where lessons could be learnt is the reflexivity tradition within qualitative research. Despite the ideal of equipoise, it is reasonable to assume that most authors of effectiveness studies do not undertake their work from a position of complete neutrality, and the same is true for authors of effectiveness reviews. Decisions about the topics, focus, and design of primary effectiveness studies and subsequent reviews, and about the outcomes that matter, the analytic strategy, and, critically, the interpretation of the data, are all value laden. Proper, rigorous reflexive accounting goes some way to reveal these biases. This is one particular area where effectiveness review methodology could benefit from longstanding practices from qualitative research and review techniques (Babones, 2016; Newton et al., 2012; Perez-Brumer et al., 2016). We would also suggest that the discussion around conflict of interest in effectiveness reviews (Cochrane, 2019b) could benefit from this qualitative tradition.
Conclusion
Qualitative evidence syntheses represent a potentially important source of information about people’s views and experiences. This evidence can help decision makers assess the acceptability and feasibility of healthcare and other services and design appropriate implementation strategies. However, methods for these types of syntheses are still evolving. Systematic reviews of effectiveness studies, on the other hand, have established methods and standards and are far more integrated into many decision-making processes.
In this paper, we have described how EPOC has developed guidance for qualitative evidence syntheses within this quantitative review environment. We have also pointed to areas where further work is needed. When developing this guidance, it has been important to display to end users that qualitative evidence syntheses are carried out with rigour and transparency. At the same time, we have strived to ensure that the quality standards we emphasise reflect qualitative research traditions. This includes acknowledging the need for methodological responsiveness and flexibility.
While qualitative evidence synthesis is a relatively new method, qualitative research in general has a well-established history, and we have pointed to areas that we believe quantitative research in general and systematic reviews of effectiveness in particular could learn from.
Decision makers increasingly turn to systematic reviews of all kinds when making decisions. This has led to an increased interest in how to develop living reviews and rapid reviews, how to encourage co-production of reviews with relevant stakeholders and how to ensure that reviews reach and are used by relevant end users. These are all areas where the systematic review community as a whole could benefit from working together and learning from each other.
Footnotes
Acknowledgements
Thanks to all of the co-authors and peer reviewers of the qualitative evidence syntheses referred to in this paper. Thanks also to EPOC’s information specialist, Marit Johansen, for her input to parts of the template described in this paper.
Author contributions
C Glenton, S Lewin, S Downe, E Paulsen, S Downe and MA Bohren conceived the template and guidance. C Glenton carried out the data extraction from reviews prepared by C Glenton, S Lewin, S Downe, MA Bohren, S Munabi-Babigumira, S Agarwal, H Ames, S Cooper, K Daniels, C Houghton, A Karimi-Shahanjarini, H Moloi, W Odendaal, E Shakibazadeh, L Vasudevan and A Xyrichis. C Glenton, S Lewin, S Downe, E Paulsen, S Downe and MA Bohren led the analysis and developed the template and guidance with written and verbal input and feedback from S Munabi-Babigumira, S Agarwal, H Ames, S Cooper, K Daniels, C Houghton, A Karimi-Shahanjarini, H Moloi, W Odendaal, E Shakibazadeh, L Vasudevan and A Xyrichis. C Glenton led the writing with input and final approval from all other authors.
Declaration of Conflicting Interests
The author(s) declared the following potential conflicts of interest with respect to the research, authorship, and/or publication of this article: The authors of this paper all have formal training in qualitative research and have educational backgrounds in sociology, social anthropology, medicine and other health sciences. Several of us work or have worked as healthcare professionals and all are researchers in public health, health systems or related fields. We all work within environments dominated by the needs and values of quantitative research and applied research. We believe that qualitative research can make an important contribution to applied sciences and can complement quantitative research by providing decision makers with valuable information about people’s needs, views and experiences. However, our experiences within these environments have shown the need to emphasise and be transparent about the processes and rigour that qualitative research involves.
Funding
The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: Development of the template described in this paper was funded by the Norwegian Agency for Development Co-operation (Norad), via the Norwegian Institute of Public Health. The Norwegian Institute of Public Health provided additional direct funding.
