Abstract
Practical guidance for navigating issues associated with combining mixed methods and case study research approaches is in short supply, particularly for novice researchers who may grapple with numerous decision points in planning and conducting such studies. This methodological discussion examines the decision-making process used in a qualitatively driven mixed methods dissertation study with a focus on how mixed methods and case study research approaches can be applied together to enhance case descriptions and interpretations. This discussion contributes to the field of mixed methods research methodology by expanding conversations about the inherent messiness of mixed methods research and by providing practical guidance to researchers interested in applying a mixed methods case study approach.
Keywords
Two roads diverged in a yellow wood, and sorry I could not travel both and be one traveler, long I stood.
A research endeavor, particularly one that investigates complex social issues, can be thought of as a journey, with the researcher acting as a navigator who, like Frost’s traveler, must make decisions along the way. These decision points may be particularly intimidating for a doctoral student embarking on a dissertation journey. After the key research questions have been determined, one of the first forks in the road the researcher faces is the choice of research methodology. For those setting out to navigate questions regarding social conditions, the array of choices is likely to include case study and mixed methods methodologies. Unlike Frost’s traveler who faced a dichotomous choice, however, the researcher at the juncture of methodological roads has another option—that of creating a new road that combines the two methodologies. Despite increasing use of the term mixed methods case study in various disciplines (e.g., Hoffman & Silverberg, 2015; Teague, Anfara, Wilson, Gaines, & Beavers, 2012), the novice researcher looking for guidance in literature will find few discussions of the tensions associated with applying these two established methodological approaches within a single study.
This article tells the story of how one researcher navigated the topography of the epistemological, methodological, and practical challenges associated with intersecting case study and mixed methods methodologies to investigate a single case. The product of the initial leg of the journey was a dissertation titled “Partners for Change: A Mixed Methods Case Study of an Intermediary-Led Partnership for STEM Education Reform” (Walton, 2014), hereafter referred to as the Partners for Change study. The dissertation journey continues through ongoing discussions among this article’s authors about the study’s implications for the dual application of case study and mixed methods methodologies. After a discussion of the team approach to this article, we describe the illustrative study, overview our thinking about mixed method case studies, and discuss how the challenges associated with intersecting the two methodologies were navigated. Finally, we offer reflections on questions associated with this methodological approach and a personal reflection on the ongoing dissertation journey.
This reflective discussion aims to contribute to the field of mixed methods research methodology by expanding conversations about the inherent messiness of mixed methods research (e.g., Freshwater, 2007; Plano Clark & Ivankova, 2016), qualitatively driven mixed methods research in general (e.g., Hall & Ryan, 2011; Hesse-Biber, 2015; Morse & Cheek, 2014, 2015), and mixed methods case study in particular (Plano Clark, Foote, & Walton, 2018; Guetterman & Fetters, 2018). This discussion also adds to the developing literature about mixed methods hybrid designs—those that intersect mixed methods with other methodologies (Guetterman & Fetters, 2018; Plano Clark et al., 2018; Plano Clark & Ivankova, 2016). Furthermore, it seeks to provide insights that doctoral student researchers may find instructive. Scholars interested in both mixed methods research and case study research can benefit from an examination of the implications of applying the two methodologies within a single study, and researchers and practitioners interested in examining cross-sector partnerships may benefit from this methodological discussion as well.
The Travelers: A Writing Team Approach
The dissertation journey described in this article represents the authors’ engagement with and ongoing discussions about ideas related to mixed methods and case study over the course of several years. The authors’ thoughts about using a mixed methods case study approach were applied to and refined by reflection on the methodological approach used in the Partners for Change study. While the dissertation journey was experienced by the first author, the ideas expressed in this article reflect the authors’ collective thinking about the study.
The lead author, Janet, began her doctoral studies with a career background in nonprofit and cross-sector partnership leadership. These experiences pressed on her the need for research applicable to partnership practice and suggested the topic of the Partners for Change study. This need, along with a focus on mixed methods research in her doctoral studies, engagement with methodological literature, and conversations with her dissertation committee members, drove her approach to the study.
Linked to Janet initially as her dissertation chairperson, Carla’s research focused on examinations of science education practices in the context of educational reform initiatives. Carla’s experiences with qualitative case studies were critical in informing the case study design of the Partners for Change study and influenced the choice of research questions and methodological approaches appropriate to answering those questions. Carla also acted as the initial liaison to the case participants and site.
The second author, Vicki worked with Janet as an instructor in her research methods courses and then later as a dissertation committee member. Vicki’s background in methodological research specializing in mixed methods designs played a key role in shaping Janet’s thinking about formulating the mixed methods approach to the study. Vicki has engaged with ways of combining mixed methods and case study in her own writing (e.g., Plano Clark & Ivankova, 2016), and Janet and Vicki’s continuing discussions over the past few years have increasingly focused on the methodological implications of mixed methods case studies.
In the midst of these discussions and growing ideas, Lori began the early design of her dissertation study, a mixed methods comparative case study. Having a common interest in the implications of a mixed methods case study design, Lori joined the team’s ongoing conversation regarding how to effectively combine mixed methods and case study approaches.
Janet’s initial engagement with and reflections about methodological literature in the planning of her dissertation study evolved into a group reflection (Siltanen, Willis, & Scobie, 2008) about how this study fit into, and potentially expanded, current mixed methods case study discussions. Based on ongoing engagement with mixed methods case study literature and their collective thinking, Vicki, Lori, and Janet advanced a framework for intersecting mixed methods and case study methodologies (Plano Clark et al., 2018). The development of this framework and consideration of others’ perspectives as reflected in methodological literature continue to inform the authors’ ideas and discussions about the intersection of mixed methods with case study in a single case design. The ideas presented here represent the writing team’s current thinking about this topic within the context of one dissertation journey. In the discussion that follows, we use the pronouns we and our to convey that joint thinking. Where Janet offers personal reflections on her study, the singular I is used.
The Journey: The Partners for Change Study
Memos and journal entries created during the dissertation process continue to inform our reflections on challenges associated with methodological choices in the Partners for Change study and represent a travel log of sorts. A brief recounting of the study incorporating these ideas is presented here. For a full description of the study and its findings, see Walton (2014).
Study Purpose
As a leader of a cross-sector partnership for science, technology, engineering, and mathematics (STEM) education reform, I developed a particular interest in how leadership structure, and especially intermediary leadership (Honig, 2004), or leadership placed within an organization outside of the partnership’s constituent groups, affected partnership work. As I transitioned to academia and delved into the literature base on cross-sector partnerships, I found that the evidence regarding how the leadership structure of partnerships influence their potential for success was thin. In addition, although practitioner-based publications such as Stanford Social Innovation Review and organizations such as the National Council of Nonprofits suggested that case studies could be useful tools for studying partnerships for social change (e.g., Hanleybrown, Kania, & Kramer, 2012), I found little guidance about designing such studies in the practitioner literature.
These literature gaps suggested that a case study of a cross-sector partnership with rigorous descriptions of research methods could be useful to both academicians and practitioners. The purpose of the Partners for Change study, therefore, was to understand how the leadership structure of a cross-sector partnership influenced its potential for success in achieving its objectives. A related objective of the study was to provide an example of methodologies that could be applied by both practitioners and academicians to examine such partnerships.
The Case
The Eastern STEM Change Partnership (ESCP) was a federally funded partnership in one region of a Southern state, organized around the goal of reducing a perceived STEM skills gap evidenced by a mismatch between workforce competencies and the demands of employers in technical and scientific fields (pseudonyms are used for the names of the partnership and other organizations associated with the case). The ESCP pursued this goal by forging more robust connections between the K-12 educational community and businesses, community-based nonprofit organizations, and postsecondary educational institutions. The partnership engaged stakeholders from each of these sectors and placed its leadership within an intermediary organization, the Regional Foundation for Education. The study explored the partnership in early 2014, its last year of public funding.
Merged Theoretical Framework
The theoretical approach to the study combined practitioner- and research-based theory as a means to incorporate the values of various stakeholders. The resulting conceptual approach merged collective impact theory (Kania & Kramer, 2011), a framework of five factors characterizing successful cross-sector partnerships, with the collaboration factors framework (Mattesich, Murray-Close, & Monsey, 2001), a set of 20 factors identified in the literature as key to successful collaboration in partnerships. This combination, hereafter referred to as the merged theoretical framework, offered a dual focus on the role of intermediary leadership and on the phenomenon of collaboration across the ESCP for the Partners for Change study.
Methods and Case Report
A single case study design was chosen for the Partners for Change study to provide an in-depth consideration (Sandelowski, 2010) of the unique ESCP leadership structure. Although a network of STEM education reform partnerships had been federally funded within several states, the ESCP was the only such partnership that had chosen an intermediary leadership structure—with leadership situated outside of K-12 or postsecondary education. Thus, the ESCP represented a unique case (Yin, 2009) in the environment of federally funded STEM education partnerships.
In order to gain a deep understanding of the context of the partnership and individual stakeholders’ perspectives, a qualitative case study approach was chosen to investigate collective impact within the partnership. The merged theoretical framework also suggested the usefulness of assessing collaboration using the Wilder Collaboration Factors Inventory, a quantitative survey instrument based on Mattesich et al.’s (2001) collaboration factors. To address the overarching purpose of the study, the case was examined by investigating qualitative, quantitative, and mixed methods research questions using interviews, document review, and the survey instrument. Table 1 provides a summary of the research questions and data collection for the study.
Research Questions and Associated Data Collection and Analysis for the Partners for Change Study.
The merged theoretical framework provided links across the methods and drove the mixed methods integration strategies. Survey results were linked to qualitative findings in a side-by-side joint data display (Creswell & Plano Clark, 2011) that identified complementary and divergent quantitative and qualitative findings (an excerpt of this display is provided in Table 2). Mixed methods integration continued through the inferences stage where quantitative collaboration-level findings were interwoven with qualitative findings in the case report. This resulted in a unified narrative depicting the ESCP through the perspectives of individuals within the broader context of partnership-wide collaboration.
Excerpt of Side-by-Side Data Joint Display for One Theme From Partners for Change Study.
Note. LOC = level of collaboration. The Wilder Collaboration Factors Inventory measured LOC using the following scale: strong collaboration, LOC ≥ 4.0; collaboration approaching strength, LOC = 3.5 − 3.9; collaboration approaching concern, LOC = 3.0 − 3.4; and collaboration levels that are cause for concern, LOC ≤ 2.9 (Townsend & Shelley, 2008).
Overall, the case narrative told the story of a cross-sector partnership that relied heavily on the personal traits of dynamic leaders to make connections across sectoral boundaries and identified areas of strength and weakness that could be used to guide this and similarly situated partnerships’ future work. Interpretation of jointly displayed findings led to identification of nuanced relationships between constructs that would not have been apparent using either qualitative or quantitative findings in isolation. For example, the dialog between quantitative collaboration findings and qualitative findings regarding collective impact themes revealed key relationships between the partnership’s failure to establish robust data collection systems and its potential to leverage funds for its long-term sustainability.
Territorial Boundaries: Negotiating a Combined Design
Our discussions about the Partners for Change study occurred in the context of a broader theoretical discussion about how boundaries between mixed methods and case study methodologies are established and how these boundaries are negotiated in mixed methods case studies (Plano Clark et al., 2018). In this discussion, we view both case study and mixed methods as methodologies, or ways of thinking about research. As such, we view case study and mixed methods as approaches that not only inform a study’s methods but also the overarching conceptualization of the study encompassing all decisions, from formulating research questions to drawing inferences (Plano Clark & Ivankova, 2016). We acknowledge that there are extensive literature bases and ongoing methodological discussions about both methodologies that touch on many complex issues. Furthermore, we recognize that both mixed methods and case study appeal to researchers working from a range of different epistemological and disciplinary perspectives. In particular, various epistemological stances, including those associated with constructivist and postpositivist worldviews, can inform both mixed methods and case study research (Plano Clark et al., 2018).
At the same time, we acknowledge that there are specific methods often associated with each approach. While case study researchers have traditionally drawn on a variety of methods, both qualitative and quantitative, case study is generally viewed as a primarily qualitative methodology with a focus on creating richly detailed case reports (e.g., Creswell, 2013; Schwandt & Gates, 2018). In contrast, mixed methods studies may have qualitative, quantitative, or equal priorities and the methodological focus is on integrating methods to generate conclusions that supersede the capacity of either method applied in solo (e.g., Creswell & Plano Clark, 2011). Furthermore, mixed methods studies may integrate quantitative and qualitative perspectives at the levels of philosophical assumptions (e.g., postpositivism and constructivism) and methodology (e.g., experiment and case study) in addition to integrating at the level of methods (e.g., survey and interviews; Fetters & Molina-Azorin, 2017).
In summary, there are many possible ways to define both mixed methods and case study research. For the purposes of this discussion, we conceptualized mixed methods research as an approach that aims to understand the complexity of a phenomenon by intentionally integrating philosophical assumptions, methodological approaches, and methods associated with quantitative and qualitative research at one or more points within the study in order to provide more holistic and nuanced insights about the phenomenon (Creswell & Plano Clark, 2011; Johnson & Onwuegbuzie, 2004; Plano Clark & Ivankova, 2016). We conceptualized case study research as the naturalistic investigation of an ecologically bound phenomenon in which the researcher gathers various types of evidence to generate insights without manipulating the phenomenon (Creswell, 2013; Sandelowski, 2010; Yin, 2009).
To further our thinking about the possibilities for how case study and mixed methods can be combined, we reviewed examples of studies that used the identifier mixed methods case study to describe study design. We found that growing numbers of published research reports (e.g., Bertling, 2015; Hoffman & Silverberg, 2015; Kaplan, Lichtinger, & Margulis, 2011; Teague et al., 2012; Weinberg, Oshiro, & Shea, 2014) and dissertations (e.g., Jong, 2009; Marchbank, 2015; Ziegler, 2016) were using this designation. We grappled with the use of these terms in such studies, since case study and mixed methods seemed to refer to methods in some study reports and methodologies in others, and we felt that this distinction was critical to understanding the methodological approaches to the studies. We also found a wide array of ways in which mixed methods and case study approaches had been combined within the examples, including case studies that incorporated multiple data sources and presented quantitative and qualitative findings in parallel and mixed methods studies that identified a study site as a case.
As we pondered these differences, our conversations focused on how key features of both methodologies were incorporated in examples of mixed methods case studies. We were convinced that the significant areas of overlap between case study and mixed methods research—including their prowess in examining complex phenomena, their adaptability to multiple disciplines and contexts, and their practical focus on combining multiple sources of data to understand phenomena—were key to understanding how these two approaches could be meaningfully applied together. At the same time, the points of divergence between the two methodologies could not be overlooked, especially since one of the key distinguishing features between the two methodologies, we believe, is their focus. Our discussions increasingly addressed how to define the boundaries between case study methodology and mixed methods methodology and how researchers can apply these methodologies simultaneously in ways that recognize and honor the traditions of each while leveraging the methodological opportunities presented by their dual application.
These ideas led us to consider more deeply the issues involved when combining both case study thinking and mixed methods thinking in a study. Our considerations were informed by Plano Clark and Ivankova’s (2016) concept of intersecting to describe the joining of mixed methods with another methodology to generate a hybrid design. In a previous article, we conceptualized mixed methods case study as an approach that brings together mixed methods integration with case study approaches to achieve deep and rich understanding of phenomena, essentially creating a mixed methods version of case study (Plano Clark et al., 2018).
Our conversations led us to conclude that the Partners for Change study represented an example of an approach in which the two methodologies were intentionally intersected, creating a methodological dialogue throughout the study that leveraged the strengths of each methodology to create a rich, holistic case description and interpretation. Using this perspective, we revisited the research process for the Partners for Change study to explore the tensions and lessons learned from applying a mixed methods approach to case study.
Bumps in the Road: Navigating the Tensions of Intersecting Methodologies
The challenges inherent in intersecting case study and mixed methods methodologies to create a mixed methods version of case study can create significant points of tension for the researcher (Plano Clark et al., 2018). In the sections that follow, we recount our experiences in grappling with tensions in the context of the Partners for Change study. These tensions included conceptualizing the philosophical foundations of the study, bridging the academic/practitioner divide with theory, situating methodologies within the study design, conveying the study design, and enhancing the case interpretation using mixed methods.
Conceptualizing the Philosophical Foundations of the Study
Both mixed methods and case study approaches are applied by researchers identifying with a wide variety of philosophical orientations (Johnson, 2017; Maxwell & Mittapalli, 2010; Schwandt & Gates, 2018; Shannon-Baker, 2016; Yazan, 2015). Thus, it is particularly important for researchers to examine their own philosophical assumptions when combining these two methodologies and to consider the implications of these assumptions for the design of a mixed methods case study.
As I delved into the topic of research paradigms in my doctoral coursework, I grappled with how to best identify my worldview within the continuum of philosophical orientations advanced in literature. I initially felt an affinity with the pragmatic worldview’s action orientation and flexibility (Johnson & Onwuegbuzie, 2004), and was particularly drawn to the pragmatic focus on the utility of findings and the implicit rejection of the notion that there is a single best way in which to understand our world. Pragmatism is undoubtedly a worldview that lends itself to intersecting methodologies and was the worldview I identified in my dissertation. However, as I continued to reflect on my experiences in discussions with the coauthors, I began to wonder whether pragmatism fully reflected the dialogic approach grounded in the merged theoretical framework.
Each component of the merged theoretical framework played a separate part in building an in-depth understanding of the partnership from micro and macro perspectives. Like actors in a play, these two elements “spoke” to one another from their different roles. The individual partners’ perspectives captured through qualitative methods created a subjectively drawn, finely grained micro picture of the partnership and its work, whereas the collaborative practices were treated as a more objective, measurable macro construct captured through quantitative methods. The associations drawn between the two conceptual bases in the merged theoretical framework allowed quantitative collaboration findings to work in dialog with findings regarding partners’ experiences and perspectives. These associations informed displays of quantitative findings organized by qualitative themes and provided opportunities for individual partner perspectives to lend insight into the macro context of collaboration within the partnership and vice versa.
These reflections led me to consider that my approach to research might be more aptly described as dialectical pluralism, a philosophy that values and balances the diverse goals of various stakeholder groups, and that continually engages in the tensions between various worldviews (Johnson, 2017). A dialectical pluralistic perspective for the Partners for Change study highlights the dialogue among practitioner and academic stances; pragmatic, postpositivist, and constructivist assumptions; and the mixed methods and case study methodologies that were foundational to the study design.
I began my doctoral studies believing that I must firmly identify with one of several categorical worldviews that had been advanced in the literature. Critical reflection on my methodological approach and decision-making led me to more deeply understand my own philosophical assumptions and to realize that these assumptions are not static but are instead influenced by an ever-deepening understanding of myself as a researcher and the complexity of societal issues. Our team discussions highlighted the idea that no single philosophical approach is a “best fit” for combining case study and mixed methods methodologies. Instead, we became more convinced that the continuum of philosophical approaches that serves as the foundation for combining methodologies is reflected in the various ways in which these two methodologies can be combined. For example, postpositivist approaches to case study might view cases as settings and emphasize replicability and the ability to draw overall generalizations rather than focusing on the nuances of the particular case. Constructivist approaches might include quantitative measures in a qualitatively driven case study for descriptive purposes with little focus on quantitative analysis or mixed methods integration. Researchers approaching mixed methods case study from a pragmatic perspective might emphasize the ways that interaction between methods can answer research questions, while a researcher with a dialectical perspective might engage more fully with tensions that arise from combining various disciplinary stances and philosophical traditions.
Bridging the Academic/Practitioner Divide With Theory
Research requires not only self-reflexivity but also an outward orientation to the needs of stakeholders that may require working across disciplinary and sectoral boundaries (Denzin, 2010). Because of my practitioner background, I believed that bridging the divide between the academic research community and the partnership practitioner community was crucial to creating a meaningful study, a belief that informed the theoretical approach to the study and the study design. While case studies incorporating multiple data sources are frequently used to investigate phenomena in the practitioner setting as well as in the academic setting, studies conducted in practitioner settings tend to include less rigorous discussions of methodology and methods and very limited discussions of academic literature, being framed instead by practical needs such as program evaluation or communication with other practitioners in similar settings. The merged theoretical framework, combining collective impact, a practical theory with roots in practitioner research, and the collaboration factors that Mattesich et al. (2001) synthesized from academic literature, therefore provided a basis for the study that would be credible and useful for both academic and practitioner stakeholders.
As a university-insider with a recent history as a university-outsider, I approached my dissertation research with a desire to craft a study that would be useful, rigorous, and valid in both the practitioner and academic communities. My practitioner experiences had impressed on me that any research or evaluation project must be resource efficient and provide results adaptable to use by various stakeholders including boards of directors and potential funders. My university experiences suggested that conducting a case study would be an extended process, that my case narrative could run into the hundreds of pages, and that my theoretical framework must be firmly grounded in academic literature.
By identifying support for practitioner-based theory in the academic literature and evidence of academic-based theory informing practitioner literature, I was able conceptualize a merged framework that would be convincing and meaningful to both groups. This merged framework helped distinguish and relate the aspects of the phenomena to be explored qualitatively and those that could be measured quantitatively. The practitioner-based collective impact theory informed the development of an interview protocol that investigated partners’ perspectives of the ESCP’s work in each of five areas. The literature-based collaboration factors framework was associated with an existing survey instrument that gauged the level of collaboration through the measure of 20 factors. Furthermore, the merged theoretical framework that related the five areas to the 20 factors provided the organizational structure for integrating the different forms of evidence to understand the case (see Table 2 for results for one of the themes in the merged theoretical framework). This use of theory to reconcile disparate stakeholder norms ultimately resulted in a case report that was firmly grounded in evidence-based theory and that also incorporated language and values meaningful to practitioners.
Situating Methodologies Within the Study Design
Another tension that emerged in the Partners for Change study was how best to position the two methodologies within the study’s design. Typology-based approaches (Crotty, 1998; Maxwell, 2013) are often emphasized in doctoral coursework because they identify prototypical designs and the decisions commonly associated with each (Creswell, 2013; Creswell & Plano Clark, 2011; Teddlie & Tashakkori, 2009). Although typologies existed for case study designs (e.g., Yin, 2009) and mixed methods designs (e.g., Creswell & Plano Clark, 2011; Teddlie & Tashakkori, 2009), I struggled to apply existing typologies when conceptualizing how the two methodologies interacted to form a mixed methods case study approach. While existing typologies are frequently inadequate to fully describe the complexity of mixed methods study designs (Guest, 2013), I found that examples of how to situate methodologies within hybrid designs were in particularly short supply. Therefore, I needed to address how best to describe the intersection of the methodologies within this study’s design, which could contribute to ongoing discussions about mixed methods case studies in particular and hybrid designs more generally.
Case study designs can be differentiated in terms of the number of cases and reasons for studying the case (Stake, 2006). Because it was the study of a single case to examine the issue of organizational leadership structure in cross-sector partnerships, from a case study perspective the Partners for Change study aligned most closely with an instrumental case study design. That is, it was a study in which a phenomenon was explored within the context of a case with the primary aim of gaining insight about that phenomenon (Yin, 2009). Mixed methods designs can be differentiated across numerous dimensions including, for example, a study’s purpose and points of integration (e.g., Greene, 2007; Guest, 2013). A common approach is to describe the design in terms of the timing and priority of the quantitative and qualitative methods (Creswell & Plano Clark, 2011; Morse, 1991). From this mixed methods perspective, the study most closely aligned with a concurrent, qualitatively driven mixed methods design because the two methods were implemented in the same phase of the research and the qualitative methods had greater importance in addressing the overall study purpose. Although these two perspectives provide accurate descriptions of the study design, they did not fully convey the relationship between the two methodologies.
In considering the design for including a quantitative measure within a primarily qualitative case study, I turned to the state-of-the-art methodological understandings in the mixed methods field which, at the time the study was planned and conducted, included an embedded design that denoted the incorporation of a secondary method within another methodology (Caracelli & Greene, 1997; Creswell & Plano Clark, 2011; Greene, 2007). Since my study included a quantitative method in a primarily qualitative case study, I adopted this terminology and identified the design as a case study variant of the embedded mixed methods design.
Although the study was designed and initially reported as an embedded design, I felt some discomfort with this designation since it seemed to downplay the role of mixed methods as a methodological approach to the study by relegating the quantitative methods to the role of secondary data source and minimizing the need for integration. The quantitative methods in the Partners for Change study, besides simply adding a secondary data source to a qualitative study, were key to understanding the case and were guided by the overall theoretical approach. It became increasingly clear in our team discussions that the embedding design did not adequately describe the interaction of the case study and mixed methods approaches.
As the writing team engaged with the question of how to conceptualize this study’s design (and others like it), our conversations were informed by ongoing discussions of this issue in the literature (e.g., Curry & Nunez-Smith, 2015; Plano Clark et al., 2018; Plano Clark & Ivankova, 2016). A key turning point was the introduction of the concept of intersecting methodologies (Plano Clark & Ivankova, 2016) into our conversations. Whereas embedding placed emphasis on using different methods within one methodology, intersecting recognizes the use of more than one methodology in a single study, a framework that resonates with my approach to the Partners for Change study.
Our team discussions led us to conceptualize the Partners in Change study as an example of intersecting mixed methods with case study. The intersecting framework captures the essence of the study as one in which methodological approaches are intentionally joined to create a dual application of methodologies within a single study. As compared with embedding, this conceptualization serves to treat the methodologies more as equal partners, and also better highlights the integrative possibilities afforded by the approaches and methods in this study.
Conveying the Study Design
In addition to the challenge of conceptualizing the study’s design, another related challenge of this dissertation journey was how to convey this conceptualization to others when writing or speaking about the study. Due to the complexity of mixed methods designs, strategies for visually conveying the components of a mixed methods study have been advanced in the mixed methods literature (Creswell & Plano Clark, 2011). These strategies include the use of a notation system and procedural diagrams. Although guidance exists for applying these strategies to various mixed methods designs, I had difficulty in applying these conventions in a way that accurately reflected the Partners for Change study’s design. Although it is not uncommon to encounter difficulty in expressing the complexity of mixed methods studies using existing conventions, I found these conventions particularly problematic for applying mixed methods thinking in tandem with case study thinking and considered new possibilities as illustrated in the following discussion.
The mixed methods notation system aims to convey the essence of a study design using letters and symbols. Morse (1991) introduced the following elements: (a) the shorthand “Quan” for quantitative and “Qual” for qualitative, (b) uppercase letters to indicate prioritized methods and lowercase letters for lesser priority, and (c) arrows (→) for sequential timing and pluses (+) for concurrent/parallel timing. Creswell and Plano Clark (2011) later introduced the use of parentheses to indicate embedding. Since I had identified my study as an embedded mixed methods case study, I used the QUAL(quan) notation in my dissertation to indicate that a quantitative measure was embedded with in a primarily qualitative study. This notation, however, did not adequately convey several aspects of my study design including the use of case study methodology, the timing of the methods, or the integrative nature of the approach. No convention currently exists for a mixed methods case study design that intersects the two methodologies. Building on our writing team’s ideas about an intersected approach to combining methodologies, I would now describe my study in the following way: MMCS(QUAL+quan), where MMCS reflects both the mixed methods (MM) and case study (CS) aspects of the study.
Procedural diagrams are tools for depicting the flow of activities within a mixed methods study, including the collection, analysis, integration, and interpretation of the quantitative and qualitative data. General guidelines exist for developing such diagrams (Ivankova, Creswell, & Stick, 2006), but these guidelines do not offer recommendations for conveying the nature of an embedded or intersected mixed methods approach. Figure 1 depicts two attempts to develop a diagram that conveyed the flow of activities in my dissertation. Figure 1a was modeled on a procedural diagram developed for an embedded experiment by Brady and O’Regan (2009, Figure 4) and was used in presentations leading up to my dissertation proposal approval. Figure 1b appeared in my final dissertation and was informed by methodological writings about the embedded design (e.g., Creswell & Plano Clark, 2011) and my desire to emphasize the case study aspects of the study. Although the clarity of the diagram improved in the transition from Figure 1a and 1b, I struggled with how to convey the importance of both the case study approach and the mixed methods integration strategies in the study’s design.
Two procedural diagrams for the Partners for Change study conceptualized as a qualitative case study with an embedded quantitative component: (a) Procedural diagram from the proposal development phase (November 2013) and (b) Procedural diagram from the dissertation (Walton, 2014, p. 69, Figure 4).
Today, looking at my study from the perspective of intersecting mixed methods and case study, I offer Figure 2 as a revised study diagram that depicts the study design using the intersecting framework. This revised diagram adopts a more conceptual approach to depicting the study than did Figures 1a and 1b, which attempted to simultaneously capture finely grained procedural details of the study’s methods while also depicting the overall methodological approach. Figure 2, in contrast, maps details of the convergent mixed methods design onto the intersected methodological approach. The equal size of the circles depicting methodological approaches reflects the equal role of the case study and mixed methods thinking that were applied to the study. The diagram permits separate articulation of the design choices informed by each approach, but indicates that these design choices contributed to a case interpretation in which approaches were interwoven to create a unified whole. The intersection of the two circles, therefore, represents the mixing not only of methods but also mixing at the level of methodologies and philosophical approaches.
A procedural diagram for the Partners for Change study conceptualized as a mixed methods case study that intersects the two methodologies.
There are undoubtedly many ways to productively depict hybrid designs such as mixed methods case studies. Figure 2 provides just one example of how a single case mixed methods case study using convergent mixed methods could be conceptualized. Other researchers using more complex data sources or those conducting multiple case studies may find additional depictions of procedural flow to be productive in their study diagrams (e.g., see Plano Clark et al., 2018). The evolution in the notation and diagram used to describe this study, however, reflects my growing awareness of the uniqueness of my study design and willingness to remain open to the idea that it may not fit current methodological conventions. This awareness ultimately led me to be able to articulate more nuanced study descriptions and also prompted our team to consider ways to contribute to methodological discussions about these hybrid designs.
Enhancing the Case Interpretation Using Mixed Methods
Describing and interpreting the interactions of individual perspectives, relationships between and within stakeholder groups, and various contextual features of a dynamic and changing partnership was a daunting challenge. The mixed methods case study approach provided a useful framework for synthesizing the various findings from the study in a way that captured micro (individual participants’) views, macro (partnership-wide) findings, and the nuanced relationships between these findings. The application of mixed methods integration strategies to case study findings led to meta-inferences that that we believe were deeper and more unified than would have been possible with a study that applied methods in parallel or that used a mixed methods convergent design without the focus on case interpretation.
The dialogue between methods was guided by the merged theoretical framework and led to key findings about a nuanced interplay of factors contributing to the partnership’s prospects for sustainability. For example, survey results indicated concern among partners about leadership in terms of resources such as funding, staff, and materials; however, these low survey scores contrasted sharply with high scores for another survey item that focused on the skills of leaders. Interview findings provided insight into these disparate survey scores for leadership, revealing that universally strong support for the partnership’s leaders was undergirded by a concern for the sustainability of leadership due to the impending cessation of public funding. Further insight into this phenomenon was provided by connecting quantitative and qualitative findings about the partnership’s metrics strategies. The ESCP’s resource limitations had contributed to its failure to establish long-term plans for collecting data that would provide evidence for the partnership’s progress toward its goals. The lack of measurement capacity had critical implications for the ESCP’s ability to secure the funding necessary to sustain the leadership it so valued, a phenomenon that contributed to a sense of skepticism about the partnership’s future. This identification of measurement systems as a critical issue in the ESCP’s potential for sustainability was made possible by the in-depth case analysis combined with theoretically driven integration strategies.
Furthermore, visual displays of findings that illustrated qualitative and quantitative findings in tandem were made possible by the use of the merged theoretical framework (see Figure 3). The framework provided a bridge between the qualitative and quantitative methods since the qualitative themes could be associated with constructs from the quantitative framework. This enabled me to display these concurrently to provide a visual representation of survey findings organized by stakeholder group and associated with interview themes. These visual displays also served as an example of how robust mixed methods approaches could provide case interpretations that could be especially useful to practitioners seeking ways to summarize data visually for purposes of communicating with boards of directors, funders, and policy makers.
Example of visual display of findings from the dissertation (Walton, 2014, p. 159, Figure 5).
Concluding Questions
The current phase of our engagement with the Partners for Change study and mixed methods case study is captured not in firm conclusions and recommendations, but rather in questions that researchers may consider in the course of their own journeys. We acknowledge that our conceptualization of intersecting mixed methods and case study is but one way to navigate the issues surrounding mixed methods case studies and that other researchers and methodologists may navigate these tensions differently, in ways that are informed by their own perspectives and experiences. The tensions shared in this article represent our thinking at this point in time, and we fully expect our discussions to continue to advance our ideas around the unique ways in which mixed methods case studies advance the goals of contextualized research. Considering the Partners for Change study in this context served to propel our conversations forward and suggested two key questions for ongoing consideration that we believe summarize the contributions of this work to the field of mixed methods research.
Contribution to the Field of Mixed Methods Research
How can novice researchers successfully navigate the messiness of the mixed methods case study journey? Published reports of research often describe research processes that are tidy and linear; a two-paragraph methods section tells the story of a study in which the researcher planned, executed, and reflected on her study without missing a beat. Since reading these study reports is a primary occupation of most doctoral students, the novice researcher is liable to enter her study with the idea that a solid plan is all she needs. In fact, the journey from the initial idea to the study’s completion is anything but linear, and continues as the researcher communicates and builds on the study’s findings. Our hope is that this discussion encourages novice researchers to not only draw on existing examples of study designs and methodological conventions but to engage actively with the messiness of the research process (e.g., Freshwater, 2007; Plano Clark & Ivankova, 2016). We believe that conversations about issues such as the need for innovations in mixed methods conventions—including terminology, notations, and diagrams—are key to propelling the mixed methods field forward and answering the call for increasingly “imaginative blends of traditional and emergent methods” (Teddlie & Tashakkori, 2012, p. 778).
Although the dissertation journey is necessarily one led by a single researcher, drawing on the experiences and insights of other researchers and methodologists can support researchers’ self-reflexivity in issues such as identifying their philosophical assumptions, positioning methodologies within a study, and conveying their study designs. Novice researchers are particularly challenged to chart their own paths, formulating uniquely personal approaches to their studies and navigating decision points that will have implications for their research and career trajectories. Our experiences suggest that a well-chosen dissertation committee representing multiple research competencies can be critical to navigating an inherently messy process. Dissertation approval is not necessarily the end of the journey; however, and post-dissertation scholars and researchers may find value in a shift to group reflexivity that brings new perspectives to the individual researcher’s work and allows for retrospective consideration of the study that can inform future methodological choices.
How can the theoretical framework influence the methodological approach to a study? Our team’s recent discussions have focused on the role that a study’s theoretical framework plays in the ways that both case study and mixed methods thinking can be applied within a single study. The Partners for Change study illustrated a mixed methods case study in which the theoretical framework served as a compass, giving direction at decision points based on the ways in which the two frameworks were connected. Thus, this study suggested that theory may serve as a useful (and potentially essential) tool to craft studies that are at once methodologically sound and that produce results that can be used for decision making in the practitioner world. Questions remain, however, about how to effectively bridge the academic/practitioner divide. For example, how can a novice researcher negotiate the demands of various faculty members’ disciplinary norms that may not value practitioner-based research as an evidence base? And, how can researchers and practitioners forge meaningful research relationships that produce results useful in both communities? Guidance on the importance and utility of using theory to inform mixed methods studies is limited (Evans, Coon, & Ume, 2011; Hesse-Biber, 2010), and further work in the context of mixed methods case studies may be helpful in moving such discussions forward within mixed methods research.
Epilogue
We are not living in a world where all roads are radii of a circle and where all, if followed long enough, will therefore draw gradually nearer and finally meet at the centre: rather in a world where every road, after a few miles, forks into two, and each of those into two again, and at each fork, you must make a decision. (C. S. Lewis)
And so the journey continues. As a novice researcher I initially sought the “right” answers to methodological questions and clear direction at each fork in the road, with the assumption that a straight path lay ahead of me if only I chose wisely. Now I look back—and forward—with the conviction that, when investigating complex social phenomena, there is no well-paved methodological expressway to understanding. As Lewis’s ever-forking road implies, the researcher builds her own road even as she is traveling on it, guided by her philosophical assumptions, past experiences, and the example of past travelers. Each journey will be as unique as the individual(s) traveling and, while well-informed methodological decisions are key to creating reliable and trustworthy conclusions, researchers will find there are few dichotomous choices on the way and that the journey is limited only by their own assumptions and understandings.
Novice researchers finding themselves in Frost’s “yellow wood” may find it useful therefore, to embrace the messiness of the research process, thinking beyond the first impression of the either/or choice of diverging roads. In this way, they will create their own unique paths mindful of the opportunities and challenges inherent in the journey, knowing that the road lies not only behind them but that they continue to build the road ahead as their ongoing reflections on philosophy, theory, and methodology highlight new ideas. We hope that this tale of one researcher’s dissertation journey will help prepare and empower other doctoral students as they navigate the roads of their own research journeys.
Footnotes
Author’s Note
Carla C. Johnson is now affiliated with North Carolina State University.
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
