Mixed Methods in Search of a Problem

Complexity theory is a multidisciplinary paradigm that aims to describe the behavior of systems that, depending on the discipline of inquiry, could be biological, social, economical, or anything else that is systemic. The theory maintains that the behavior of systems needs to be understood in terms of that of its interacting components (cells, individuals) as well as in terms of the system as a whole (tissue, the human body, social organizations), and that the interaction between components within the system does not necessarily follow a predictable course (Waldrop, 1992). The theory derives much of its appeal and explanatory power from the applicability of a set of simple principles about how systems work to a wide range of phenomena, and from the possibilities it creates of looking differently at existing scholarly problems, including the wicked ones. Examples of some fruitful applications in the applied social sciences are the extensive study of bottom-up interactive processes within for-profit organizations (team formation and the local identification of leaders), which has changed our views about organizational efficiency (McKelvey, 2004), and the use complexity theory to better understand areas of inertia in the public school system in the United States (Koopmans, 2016). Like mixed methods, complexity theory provides an angle to address the shortcomings of conventional research paradigms such as randomized control trial and quasi-experimental designs, which focus on intervention outcomes without regard of the implementation story, or ethnographic research that does not investigate the causality generated by the nested structure of social systems.

The Mixed Methods Task Force Report rightly notes that a mixed methods approach is inherently suitable to address complex problems or phenomena and investigate the interplay of factors operating at the individual, relationship, community, and societal levels. Complexity theory provides a framework for talking about this interplay by distinguishing the behavior of individuals from that of the larger systems of which they are part. W. Ross Ashby, one of the spiritual fathers of cybernetics, argued that whenever individuals interact, they form a system, and consequently, we need to understand behavior involving interaction in terms of such larger systems. However, there is no straightforward correspondence between these different levels of description. Instead, complexity theory postulates a dynamical interplay between the behavior of individuals and that of the systems of which they are constituent parts (feedback loops). As such, the interactive behavior of individuals shapes the system, while the system, in turn, delineates the behavior of individuals within it. The behavior of a system in its entirety can therefore not be readily reduced to that of its individual components (Ashby, 1962/2004). In other words, the whole is greater than the sum of its parts.

It follows from the aforementioned insights that the multiple layers of the system require separate scrutiny, but within a methodological framework that enables one to also describe the interplay between these different levels, that is, the feedback loops. Defining systems this way thus creates a need for research that collects and analyzes data at different levels of granularity (individuals, groups, societies) and is therefore a fertile breeding ground for mixed methods research. In the social sciences, examples of this type of research from a complexity perspective are few and far between at this point (but see, e.g., Dooley, Kiel, & Dietz, 2013), so there is much room to grow.

Where complexity research differs from the future of mixed methods research as outlined in the Task Force Report is that it has a substantive agenda, often defined as the investigation of processes of stability and change in systems. The benefit of having such an agenda is that it relieves scholars from the unenviable task of having to discuss the merits of broadly stated methodological mandates, such as the desirability of using mixed methods (or randomized control trial, for that matter) in the absence of information of exactly what issues these designs are meant to address. It is important but not enough to say, as the report does, that mixed methods research derives its utility from the search for the greater good of social justice, environmental sustainability, and so on, because if we define the problems that justify our research too broadly, the ensuing methodological discussion fails to provide the direction or inspiration that is needed to develop an effective response to those challenges.

Clearly, a “third force” is emerging in the epistemological and methodological debates in the applied social sciences now that the “quantitative versus qualitative” dichotomy has outlived its usefulness, and we are at liberty to examine the crossovers. The complexity argument is but one way of justifying why such an integration of perspectives is needed. Ultimately, however, the field of mixed methods is faced with the much bigger charge of formulating an agenda for the future. Providing greater specificity about what this agenda might entail most likely would enhance the impact of this report.