Re-Envisioning Evidence Gap Maps With Qualitative Research

Systematic Review

Although the focus of a systematic review may take many different forms, for the sake of argument, the one being considered here is an effectiveness review. The process for such a review begins with a very precise question in a PICO (Population, Intervention, Comparison, Outcome) format. Afterward, searches for appropriate literature are conducted in a systematic manner in specific databases using precise search terms. Usually, two or more researchers independently choose the articles to be reviewed, and chosen articles are placed on an evidence table. The articles are then appraised using standardized instruments. Then, data are extracted and analyzed using other standardized instruments, resulting in a hierarchy of graded evidence. Such evidence is then considered by practitioners for possible changes in practice (Higgins & Green, 2011; Robinson & Lowe, 2015). Thus, the systematic review is considered to be less subject to bias that may be found in the review of the literature because the searches are less random and the articles are appraised and analyzed using standardized instruments instead of a particular individual’s viewpoint. However, even systematic reviews have not totally addressed the problem of exhaustive searches for the best evidence. According to Snilstveit, Vojtkova, Bhavsar, Stevenson, and Gaarder (2016), there are challenges to these reviews since they are “scattered across individual journals, libraries, and web sites. [Furthermore] they are often presented in inaccessible formats and are of variable quality” (p. 120). Therefore, new systematic approaches to finding the best evidence have been developed, including evidence gap maps (EGMs).

Evidence Gap Maps

One purpose of an EGM is to provide a user-friendly visual representation of systematic reviews and high-quality impact evaluations for policy makers to quickly explore the best evidence to make decisions. Another purpose of the EGM is to determine areas in which there is a paucity of research that justifies funding of new studies (Snilstveit, Vojtkova, Bhavsar, & Gaarder, 2013). The visual matrix of cells includes rows indicating interventions and columns indicating outcomes. The columns are arranged “along a causal chain from intermediate outcomes to final outcomes and cost effectiveness” (Snilstveit et al., 2013, p. 9). Within the cells are bubbles of differing colors and sizes that indicate the number and quality of systematic reviews, impact studies, and primary research articles. Upon clicking the bubble, the reader is linked to peer-reviewed summaries that are tailored to stakeholders who may not be familiar with the language of research. The creation of such a matrix requires a specific methodology that is similar to that of a systematic review yet different in certain aspects.

Evidence Gap Map’s Methodology

The methodology for developing an EGM consists of five steps, including the following: develop an intervention/outcome framework, set study inclusion criteria, search for relevant studies and assess inclusion, data extraction and critical appraisal, and analysis and visualization (Snilstveit et al., 2016).

Re-Envisioning EGMs to Include Qualitative Research

EGMS are designed to be a useful tool for “development decision makers, researchers and donors looking for evidence to inform program investments and identify where there is an urgent need for more research or rigorous evaluation” (International Initiative for Impact Evaluation [3ie], 2018, p. 1). However, how can these maps reach their full potential and be used to identify legislative and investment goals when they do not provide the context surrounding the success of described interventions? Beyond answering the questions of what and where related to program effectiveness, key decision makers must just as importantly consider the how, why, and most importantly, for whom. Interventions are not created equal and neither is their efficacy. Any nurse researcher could explain to decision makers that the context surrounding the introduction of new interventions has an enormous impact on the overall success of a programmatic offering.

A major drawback of the current structure and inclusion criteria for EGMs is the limited type of studies that are used. The exclusion of qualitative studies is glaring. When examining the efficacy of systems-level policies developed to improve population health, policy makers cannot in good faith make educated decisions without proper consideration for the contextualization of study findings that qualitative works provide. One must keep in mind how to best improve the health equity of all marginalized populations that benefit from policies and programmatic changes. Therefore, the context of what enables programming to be successful and for whom must be considered. Policy makers must be encouraged to make decisions taking into consideration the phenomenological experiences of human lives their programmatic decisions will impact first hand.

Let us envision what an EGM may include if qualitative work is incorporated the same way that systematic reviews and interventions are listed. Currently, the 3ie website has EGMs that have been completed, the design of which is fully interactive so that hovering over any colored bubble causes a pop-up window to come into view noting the details of the bubble with links to studies within that category. There is also a legend provided that notes five specific categories including impact evaluations and four categories of confidence in the conclusion of the effects of various systematic reviews. The four areas of confidence are determined by using a standardized checklist adapted from the SURE Collaboration (2011), which divided into high confidence, medium confidence, low confidence, and protocols.

The size of the bubble is indicative of the quantity of studies within that categorization inferring that there is greater strength in confidence. In keeping with the clear visual aids and quantification of confidence in the research findings, qualitative research could be designated by a separate purple triangle that could grow in size proportional to the strength of the qualitative evidence supporting the relationship between the intervention and outcome (Figure 1).

OPEN IN VIEWER

Figure 1.

Proposed Structure of Evidence Gap Maps Integrating Qualitative Studies.

Utilizing Daly and colleagues’ (2007) hierarchy of qualitative evidence would be an appropriate manner to indicate the strength of the qualitative evidence. The purpose of the creation of this visual hierarchy was to improve the ability of qualitative research to be integrated into evidence-based practice and policy decision making. The qualitative hierarchy of evidence-for-practice consists of four distinct levels in a pyramid structure. Level IV across the bottom represents single case studies that provide a singular perspective of a phenomenon of interest. Level III directly above the bottom rung represents descriptive studies that describe a phenomenon within a defined group but do not report a full range of responses or include a diversified sample. Level II, second from the top, represents conceptual studies in which theoretical principles guided sample selection. These conceptual studies mindfully include diversity of participant views; both the majority and minority divergent views are represented, but some themes for analysis are yet to be fully described. Finally, Level I, positioned at the top of the pyramid, represents generalizable studies characterized by integration of theory and literature in their study design, intentional design, and analysis measures to capture diversity of experience with clearly articulated findings that give “clear indication for practice or policy or critique with indicated direction for change” (Daly et al., 2007, p. 46). These hallmark qualitative studies would be represented by the largest purple triangle since they are most appropriate for informing decision making in policy matters (Figure 2).

OPEN IN VIEWER

Figure 2.

Proposed EGM Structure Detailing Qualitative Hierarchy of Evidence.

EGMs and Nursing Research

Beyond the exclusion of qualitative studies, the largest concern with the advent of the EGM is that it does not generally include nursing research. The cited studies in the EGMs are generated from public health, epidemiology, and the discipline of medicine. The omission of nursing is blatant. Nurses’ research is known for its easy translation for dissemination and application by decision makers both in healthcare systems and legislature. A review of the literature to date suggests that there have yet to be any known published EGMs on topics that have a great deal of nursing research. It is owed to the profession, study participants, and the greater community to collaboratively come together to publish EGMs integral to nursing research topics. This undoubtedly could be used as a powerful tool in creating the next iterations of public health policy reforms. After all, what is more meaningful and fulfilling than nursing research that leads to systems-level change in policy and practice?

To date, the 3ie EGMs center around global public health initiatives including EGM on social, behavioral, and community engagement interventions; adolescent sexual and reproductive health; and intimate partner violence prevention, each generally funded by an organization or donors (3ie, 2018). There is a place for nursing organizations including Sigma Theta Tau International, the Institute of Child Nutrition, National Institute of Nursing Research, Robert Wood Johnson Foundation, and American Public Health Association (APHA) Public Health Nursing to spearhead and financially invest in such evidence-building efforts.

The 3ie EGM platform is now available for approved users including researchers and clinicians to create their own EGM at no charge under a Creative Commons agreement. It is urged that nursing researchers build collaborative networks to crowdsource the knowledge necessary to create EGMs to help inform legislative policy and improve the return on research investments in programs for vulnerable populations. Timely topic areas of paramount interest to nursing science and the advocacy work central to our role are numerous. EGMs should be constructed for classic areas within our discipline, such as caregiving, chronic illness, cognition, gerontology, palliative and end-of-life care, primary care, and sleep. Creation of EGMs by nurse researchers could also go a long way in advocating for the most pressing public health nursing issues of today, including but not limited to migrant and immigrant health, gun control and violence prevention, improving health equity of marginalized communities including people of color and LGBTQ, and interventions to address the ongoing opioid crisis.

Conclusion

EGMs represent an exciting new frontier in the collection and dissemination of research findings to inform policy decisions and funding priorities by public and private institutions alike. EGMs will not reach their full potential without inclusion of qualitative research. Development of EGMs centered on core nursing research areas and timely public health nursing areas are both warranted and essential for dissemination of nursing research and integration into public policy decision making.