Using Sandelowski and Barroso’s Meta-Synthesis Method in Advancing Qualitative Evidence

Conceiving the Qualitative Research Synthesis

The first step of the QRS refers to conceiving the synthesis of the qualitative study. A research question addressing a clearly defined problem must be formulated. Researchers have to address a research purpose, consider available resources, decide on the target phenomenon of the study, and establish inclusion and exclusion criteria (Sandelowski & Barroso, 2007). Thus, conceiving a study is more than “getting started” as Noblit and Hare (1988, p. 26) stated as the first step in their meta-ethnography; it is far from a straightforward process (Atkins et al., 2008). Our research team consisted of nurse researchers from three European countries (Denmark, Norway, and Germany) with varying experience in qualitative research and systematic reviews. Each member and her institution contributed time, knowledge, and funding. During regular workshops, we discussed research perspectives, theories, and definitions that directed our study phenomenon (Fegran et al., 2014). The target phenomenon—“patients’ experiences of hospital transfer”—was the common research focus because some team members had previously recognized a need to reorganize their clinical practice concerning patient transfer. Other team members were involved in research programs concerning the care of adolescents in health and illness, and some were establishing research programs focusing on patients’ hospital transfers.

Deciding on a theoretical framework was an epistemological challenge. We continually found ourselves discussing our target phenomenon, “hospital transfer,” and its relationship to the concept of transition, and we became familiar with transition theories with different interpretations of the concept. In their transition theories that they developed over several decades, Meleis and colleagues see transition as an inner experience and a personal development through an ongoing process (Im, 2011; Meleis, 2010; Schumacher & Meleis, 1994). In the Meleis school of thought, transfer is a step or a critical point in the transition process. In contrast, in the seminal work of Blum and colleagues (1993), transition and transfer are equal. Transition is considered an external experience; it is a transfer, for example, from one hospital unit to another. In our studies, we followed Meleis’ understanding of transition as an inner experience and a process, and we considered transfer to be a critical point in this transition. In the conceptualization stage, we discussed our understanding of transition and transfer, and we adjusted the aims, research questions, and definitions.

In this early phase, we were similar to previous researchers in that we learned that QRS research is ambiguous. Some authors prefer to tackle this with a more formal “concept analysis” to clarify their concept by using, for example, the Walker and Avant (2005) methodology. McCormick and colleagues (2003) found it challenging to follow Noblit and Hare’s (1988) meta-ethnography when processing a meta-synthesis of their own ethnographic studies. They experienced the process as iterative and messy but acknowledged the back-and-forth process. Our experiences coincided with those of McCormick and colleagues. We realized that establishing the theoretical and methodological grounding of our QRSs was a back-and-forth and often disordered process that needed repetitive considerations among the team members.

Searching for and Retrieving Qualitative Research Reports

The QRS process continues with searching for and retrieving research reports to include (Sandelowski & Barroso, 2007). With the significant growth of published QRSs and a demand that the studies meet high standards of validity, researchers recognize the need for rigorous literature search methods (Thomas & Harden, 2008; Tong et al., 2012). The search and retrieval processes involve a number of parameters such as target topic, sample, time, and method. Search discussions concern issues such as electronic and manual searching methods, search terms, management of the information, determination of the relevance of retrieved studies, and the use of analytic devices and displays. To identify all relevant findings pertaining to the study aim, Sandelowski and Barroso suggest searching for master’s theses and doctoral dissertations because these sources often contain detailed descriptions of findings. They ask reviewers to be frank and flexible and to allow for “berry picking,” meaning retrieving reports by means other than structured, systematized literature searches (Sandelowski & Barroso, 2007, p. 41); they also recommend that reviewers iterate the search procedures and not allow information technology to substitute for manual searches.

In our QRS studies, we appointed team members to be reviewers and literature searchers. To achieve exhaustive searches, we systematically searched a number of electronic databases (CINAHL, PubMed, Embase, Scopus, PsycINFO, and Web of Science). We searched for data backward in the references of included reports and forward in “cited by” columns of databases providing this feature (i.e., Scopus, CINAHL, Web of Science, and Google Scholar). To validate the searches, we involved a health sciences librarian in the search and retrieval processes. Librarians are included as formal team members with increasing frequency (Flemming, Graham, Heirs, Fox, & Sowden, 2013); they facilitate collaboration and education, keep records, solve technology problems, and write methodologies (Dudden & Protzko, 2011; Hallyburton & John, 2010; Hopkins, Callister, Mandleco, Lassetter, & Astill, 2011). Initially in the search and retrieval phase, we developed a broad, structured, systematic search strategy with the search terms transition, transfer, experiences, and qualitative. We later narrowed the searches and included patient as a search term. Throughout the process, we kept track of retrieved reports in tables, matrices, extraction sheets, and tabular displays, some of which we included in our publications. We located new reports through berry picking in articles we knew of as a result of our insight in the research field, or we found articles through reading research literature in general.

The identification of articles through berry picking and cited by citation searches was characterized necessary because some articles, despite thorough search strategies, did not turn up in the structured literature searches. Often such reports did not use standardized keywords (e.g., Medical Sub Headings terms or database-registered keywords as in CINAHL). Therefore, the team members who performed the systematic searches did not identify them. As a research team, we expected to retrieve more relevant items from the systematic searches than was the case. We found 11 reports through the structured systematic searches and 3 reports through cited by citation searches in one study (Uhrenfeldt et al., 2013), and 17 reports from structured systematic searches and 3 reports from cited by citation searches in the other study (Fegran et al., 2014). Our search results thus confirmed the findings of previous commentators regarding the challenges of exhaustive searches (Papaioannou, Sutton, Carroll, Booth, & Wong, 2009; Wu et al., 2012). This finding might have been a result of the descriptive nature of reporting titles, unclear descriptions of abstracts, or insufficient keywords used in the primary reports rather than a result of an imperfect search strategy (Papaioannou et al., 2009; Wu et al., 2012).

In an early stage, we agreed on an electronic reference management tool to keep track of references. Using an electronic reference management system (e.g., RefWorks, EndNote) was paramount for us, and keeping track of searches, including citing citations and reports, was an asset when trying to include all the relevant literature. We acknowledged that an important threat to validity is the failure to conduct a sufficiently exhaustive literature search (Sandelowski & Barroso, 2007). Other authors have shared this attitude (Thomas & Harden, 2008); however, meta-ethnographers might have a different view on this issue. In their meta-ethnography, Noblit and Hare (1988) suggested a systematic but not necessarily exhaustive search process: “Unless there is some substantive reason for an exhaustive search, generalizing from all studies of particular setting yields trite conclusions” (p. 28). An analysis of the effect sizes in our studies showed that we would have lacked important information about the target phenomenon if we had refrained from using comprehensive searches. We trusted that we contributed to data saturation (Morse, Barrett, Mayan, Olson, & Spiers, 2008) by using citing citation searches and with it, increased the validity of our studies.

Appraising Qualitative Research Reports

After the inclusion of reports, the QRS proceeds with critical appraisal and evaluation of the individual reports followed by a comparative appraisal and evaluation across reports (Sandelowski & Barroso, 2007). Individual appraisal is conducted through several readings of each report to become acquainted with the content, find methodological strengths and flaws, and locate target findings. Comparative appraisals across reports allow reviewers to identify reports that derive from identical samples, determine what information is missing, and elaborate on displays and summaries (Sandelowski & Barroso, 2007). Following QRS guidelines, the main purpose of the individual appraisal of reports in our metasyntheses was twofold: to explore whether the studies met the inclusion criteria and to appreciate and evaluate methodological and substantive strengths and weaknesses.

We found that there are several appraisal tools to choose between and little consensus on which tool is the most appropriate (Hannes & Macaitis, 2012; Sale, 2008). After consulting Sandelowski and Barroso’s appraisal guidelines and two unrelated independent instruments—a checklist developed by the National Institute for Health and Clinical Excellence (NICE; 2012) for qualitative studies and the Critical Appraisal Skills Programme (CASP; 2013)—we decided to apply the Joanna Briggs Institute (JBI) Qualitative Appraisal and Review Instrument (QARI; JBI, 2014). The QARI is accessible online and is a coherent tool with clear questions using a 4-point scale (yes, no, unclear, not applicable; Hannes & Lockwood, 2011; Hannes et al., 2010; Newton, Rothlingova, Gutteridge, LeMarchand, & Raphael, 2012). A strength of this instrument was its brevity (only 10 questions). Other strengths were the inclusion of questions about rigor, considerations of the reflexive nature of qualitative research (i.e., questions about the location of researchers culturally or theoretically), and inclusion of researcher reflections on their own relationship to the setting researched. The question about whether participants and their voices were adequately represented was decisive for report inclusion in our studies. This consideration helped us focus on our target phenomenon (i.e., whether report data presented patient experiences as expressed by themselves and not through others such as parents or health care professionals). Adhering to Sandelowski and Barroso’s advice, no report was excluded because of poor quality. A few reports were excluded because they did not adequately report the patients’ experiences (Fegran et al., 2014; Uhrenfeldt et al., 2013).

The appraisal step was a clear eye opener for the team. During the appraisal process, we developed sensitivity for epistemological, methodological, and ethical details in the primary reports. We learned that the team model increased the quality of the appraisal, and we agreed with Sandelowski and Barroso (2007) and others (Carroll, Booth, & Lloyd-Jones, 2012) that although judicious review is an essential part of the systematic review, it has limitations based on the quality of the original study, the appraisal instrument, and the reviewers’ interests. We also learned that critical qualitative appraisals are more than checking that reports follow defined rules. Appraisal is not conducted to provide technical fixes (Barbour & Barbour, 2003); appraisal is a reflexive dialogue making congruity, legitimacy, and relevance essential to include reports (Stige, Malterud, & Midtgarden, 2009; Walsh & Downe, 2006). Thus, our prime consideration in appraising was to gain an understanding of each report before proceeding to the steps of comparing and integrating findings.

Classifying Findings in Qualitative Research Reports

Qualitative research reports differ in regard to presenting findings because of varied methodological approaches. Thus, classifying findings is vital (Sandelowski & Barroso, 2007). Sandelowski and Barroso (2007, p. 138) warned reviewers not to exclude any type of finding presentation. Together with conceptual descriptions or interpretive explanations, surveys, which many reviewers would exclude, might contain valuable data. A research report is based on the primary researchers’ decisions on including, excluding, and interpreting findings, and these decisions might transform the original data to varying degrees. The reviewers’ judgments of reports and classification of findings depend on the primary researchers’ reported information. Sandelowski and Barroso refer to the notion of first-, second-, and third-order constructs (Schutz, 1967). The participants in the primary study are the first-order interpreters, the authors of the primary study are the second-order interpreters, and the reviewers are the third-order interpreters (see also Atkins et al., 2008; Britten et al., 2002).

We focused on the content and form of the data sources when classifying the findings. Most reports included in our QRS were thematic or interpretative; some were descriptive with findings presented close to participants’ own words, and some findings were collapsed into broad abstractions in the format of paraphrases. In the classifying process (which was closely connected to the appraisal phase), one epistemological challenge was deciding what to include as data. The team had lengthy discussions related to the target findings. The aims of our syntheses were to summarize and synthesize qualitative reports about patients’ experiences of transfer. We wanted to present the study participants’ reality as represented by the primary researchers. We therefore became third-order interpreters of the first- and second-order interpreters (Schutz, 1967).

In discussing the first-, second-, and third-order issue, we improved our process for distinguishing the primary study findings that informed our focused research question. Because our research questions were focusing exclusively on patients’ experiences, we extracted only findings from the results sections. We included quotations by participants who took part in the primary studies (first order) and paraphrases or observations offered by the authors of the primary studies (second order). We could have focused more precisely on what data to extract if our aims had been formulated more precisely (i.e., “to summarize and synthesize the primary researchers’ description and interpretation of patients’ experiences of hospital transfer”). Using Sandelowski and Barroso’s methodology, previous reviewers (Duggleby et al., 2012; Lindahl & Lindblad, 2011) have also disregarded the primary researchers’ interpretation in favor of the patients’ own first-order experiences. One reason for this confusion could be that in classifying the findings, we were sometimes uncertain whether findings were derived from research participants directly. This challenge was especially apparent when the authors interpreted research participants’ views in paraphrases with no quotes or few quotes.

Synthesizing Qualitative Research Findings Into Metasummaries

Synthesizing qualitative research findings into metasummaries is one of the characteristics of the Sandelowski and Barroso’s QRS approach. Metasummary is the extraction, separation, grouping, and abstraction of text findings into numbers and statement sets (Sandelowski & Barroso, 2007). Contrary to other QRS methodologists (Major & Savin-Baden, 2010; Noblit & Hare, 1988; Paterson et al., 2001), Sandelowski and Barroso’s method supports the use of counting and numbers in the construction of metasummaries. They stress that integrating counting into the analysis is powerful. Using numbers is favorable in searching for patterns and hypotheses; it also assists in sharpening the focus and verifying analytical moves such as recognizing data patterns and deviations from data patterns. Presenting numbers in intra-study intensity effect sizes and inter-study frequency effect sizes adds to study validity. The calculations offer reviewers a quick overview of patterns and stimulate the testing of hypotheses (Sandelowski & Barroso, 2007).

One of the issues in our metasummary step concerned which approach to use in integrating findings. Inspired by Lindahl and Lindblad (2011), we decided to use approaches that were familiar to us. In one meta-synthesis (Uhrenfeldt et al., 2013), we applied content analysis (Graneheim & Lundman, 2004); in the other (Fegran et al., 2014), we approached data according to Ricoeur’s (1976) text interpretation theory: naive reading, structural analysis, and critical interpretation. In both studies, we categorized and thematized target findings into a main theme and subthemes. Furthermore, in one study (Fegran et al., 2014), we ordered target findings using the standard qualitative software package NVivo Version 9 (QSR International, 2010). We were aware of the critique against using NVivo that claims that it might cause the reviewers to purely focus on technical aspects (Bergin, 2011). Nevertheless, NVivo gave us an overview of a large data set, allowed for comparability, and supported us in calculating effect sizes. Thus, NVivo provided valuable means for further interpretation.

We found that integrating findings into metasummaries was not only a technical issue but also an iterative process whereby the research team moved toward consensus in the analysis before moving on to the final meta-synthesis step. Counting helped us focus on similarities, differences, strengths, and variations in gender, number of participants, origin of countries, and methodologies. Calculating effect sizes increased our understanding of the power of the reports and the relationship between themes and individual reports; it also assisted us in generating theory through team discussions about the summaries.

Synthesizing Qualitative Research Findings in Meta-Synthesis

While Sandelowski and Barroso (2007) defined qualitative metasummaries as “quantitative, oriented aggregations of qualitative findings that are themselves topical or thematic summaries or surveys of data” (p. 151), they claimed that a qualitative meta-synthesis is more than summaries. The final product might be an abstract integration of findings obtained in different ways such as via taxonomic analyses (Spradley, 1979), constant targeted comparison (Strauss & Corbin, 1990), reciprocal translation (Noblit & Hare, 1988), or synthesis of “in vivo” and imported concepts or an event timeline (Happ, Swigart, Tate, & Crighton, 2004; Sandelowski, 1999). An event timeline refers to placing events of interest in temporal relation to each other to elucidate how reports relate to each other. An event timeline can help reviewers ascertain the strengths of the temporal linkages (Sandelowski & Barroso, 2007). In analyzing the experiences of transfer among adolescents and young adults, we developed a figure showing the relationship between age and transfer (Fegran et al., 2014). This figure added to our understanding of the appropriate age for transfer from pediatric to adult care.

The ultimate goal of a reciprocal translation is to retain the uniqueness of the primary findings even when synthesized (Noblit & Hare, 1988). In QRS, the reciprocal translation is an integration of two modes of synthesis. In part, reciprocal translation is an integrating process; it combines in vivo concepts with an imported concept borrowed from previous studies or other disciplines (Sandelowski & Barroso, 2007). In our reciprocal translation processes, we retained the uniqueness of primary findings somewhat differently. We located “limbo” as an abstract integration of findings in the adolescent reports. Adolescents with chronic illness who were in transition from the pediatric to the adult setting were moving from the familiar to the unfamiliar; they felt as being in-between, nowhere, in limbo. In the study of patients’ experiences of intra- or interhospital transfer, we presented the final synthesis based on the reciprocal translation as an argument for nursing actions. The synthesized evidence revealed that hospital transfer should be safe, predictable, and individual.

In our studies, Sandelowski and Barroso’s distinction between metasummary and meta-synthesis deepened our methodological notions. As the demarcation between the two is somewhat vague, we urge future reviewers using QRS to elaborate on this issue. Sandelowski and Barroso were unclear about what constitutes reciprocal translation. They suggested reciprocal translations to be interpreting in vivo statements and imported concepts. In their meta-synthesis of mothers with HIV, they imported the concept “virtual identity” from Goffman (1968), and they combined this concept with statements from the primary statements (in vivo) in a double-helix-like synthesis called “virtual motherhood.” Because our metasyntheses included themes and abstract concepts (such as “being in limbo”), we discussed whether our work should be classified as a qualitative metasummary or a qualitative meta-synthesis. We believe there is a gray zone between the two; thus, we learned that the final meta-synthesis work is a real challenge (Atkins et al., 2008). We anticipate that any meta-synthesis elaboration will gain from discussions about the gray zone between metasummary and meta-synthesis and about whether an imported concept would add to the endpoint of the synthesis.