A Performative Paradigm for Mixed Methods Research

Abstract

This article introduces a performative paradigm for mixed methods research based on Andrew Pickering’s performative view of research. This paradigm is compared with three existing mixed methods paradigms: the dialectic stance, critical realism, and pragmatism. The performative paradigm assumes the existence of multiple realities that can be known in various ways. In a research inquiry, researchers create new, unknown worlds, which they subsequently investigate. Through research, an ever-higher degree of objectivity is obtained in successive “mangles,” rounds of feedback, in which the researcher listens to the created world’s feedback and adapts his or her research models accordingly. Across and within research inquiries, many and various kinds of mangles can be combined, including mangles qualitative and mangles quantitative in nature.

Keywords

performative paradigm mixed methods research objectivity mangle of practice

This article introduces a performative paradigm for mixed methods research. The development of this paradigm started with four existing, fruitful ideas regarding reality, realities, their role in scientific inquiry, and my feeling that there was no one paradigm that provided room for these four ideas in combination. The first idea is that of mixed methods researchers as pragmatists who set aside ontological and epistemological issues (Bryman, 2007). The second idea refers to the important role of a researcher’s beliefs, assumptions, and values in understanding research and its outcomes (Lincoln, 2010). The third is the idea that people live, in part, in different worlds rather than having different views of the one and same reality (Nicholls, 2017; Yin, 2016). The fourth idea is that the research process can be seen as a process in which reality “talks back” to the researcher (Pickering, 1995).

The first part of this article discusses the characteristics, possibilities, and limitations of three existing paradigms for mixed methods research: the dialectic stance, critical realism, and pragmatism. In the second part of this article, the performative paradigm is introduced and its characteristics, possibilities, and limitations are explained in detail. This paradigm is based on the notion of performativity, as developed by philosopher of science Andrew Pickering (1995). It is not related to performativity in the sense of the power of discourse to constrain what it describes (Butler, 1993).

The Dialectic Stance

From a dialectic stance, mixed methods inquiry is viewed as a dialogue between paradigms. Thus, although the dialectic stance will be called a paradigm in this article, it could also be called a “meta-paradigm” (Johnson, 2017). The dialectic stance is described by Greene (2007) as follows:

Paradigms are constituted by sets of interconnected philosophical assumptions regarding reality, knowledge, methodology, and values. The assumptive sets of different paradigms are different in important ways, but paradigms themselves are historical and social constructions and so are not inviolate or sacrosanct. Paradigmatic assumptions importantly guide and direct practical inquiry decisions, along with context and theory. Important paradigm differences should be respectfully and intentionally used together to engage meaningfully with difference and, through the tensions created by juxtaposing different paradigms, to achieve dialectical discovery of enhanced, reframed, or new understandings. (p. 69)

Greene (2007) provides a real-life example of mixed methods research, in which two paradigms, each with a distinct view of reality and how it can be known, are brought into dialogue: a constructivist view, in which multiple realities exist that can be known in various ways, and a postpositivist view, which assumes the existence of one reality that can be known objectively. In this example of dialectic mixed methods research, the postpositivist aim of obtaining objective, timeless knowledge about the state of reality is brought into dialogue with the constructivist aim of obtaining an intersubjective understanding of multiple and varied realities. The two views on what reality is (or realities are) give rise to two distinct research methodologies, each characterized by its own assumptions, aspirations, and research methods. Greene (2015) describes this for the real-life example as follows:

With its realist assumptions and its aspirations for inferences that are generalizable, postpositivism generally calls for structured, quantitative methods and data that support the comparative causal inferences desired by the philanthropic funder of the program. With its interpretive assumptions and its aspirations for inferences that represent understanding of contextual meaningfulness, constructivism generally calls for unstructured, qualitative methods and data that represent the evaluation team’s commitments to contextual understanding and honoring of the diversity of experience. (p. 611-612)

These distinct research methodologies require that research be performed for each methodology separately in a distinct research strand, before their findings are brought into dialogue:

With respect to methodologies, the mixed evaluation team selects a quasi-experimental methodology for the postpositivist strand of the evaluation and mini-case studies for the constructivist strand. (Greene, 2015, p. 612)

In order for a mixed methods researcher or research team to be able to bring two separate research strands with different views and methodologies into dialogue, they must be somehow able to alternately adopt these views and switch between views. Such a conception, however, gives rise to a number of problems. First, switching between two distinct methodologies is not what researchers typically experience when they are conducting mixed methods research. Indeed, rather than describing themselves as researchers alternating between different ontologies and epistemologies, the mixed methods researchers interviewed by Bryman (2007) depict themselves as pragmatists who set aside epistemological and ontological issues.

A second problem is that for researchers, their ontological and epistemological views are deep-rooted stances (Lincoln, 2010). Although deep-rooted stances may change over time, they typically take time for one to abandon and cannot easily be reversed, making it unlikely for an individual researcher to switch between these stances within a research project.

A third question is how the results of the distinct methodologies applied in distinct research strands are finally combined into one integrated whole. The combination of results that have been obtained using different research methods or methodologies is a fundamental characteristic of mixed methods research that is known as “integration” (Bazeley, 2012; Bazeley & Kemp, 2012; Fetters, Curry, & Creswell, 2013; Maxwell, Chmiel, & Rogers, 2015; O’Cathain, Murphy, & Nicholl, 2010). For the dialectic-stance paradigm, integration is a tricky problem, as the mixed methods research process demands that results and conclusions from different methodologies with different assumptions, ontologies, and epistemologies be brought together. However, the question is how this can be possible given their distinct origins. Another question is what view of reality applies to the integrated result. That is, what assumptions, ontology, and epistemology underlie the integrated result?

Finally, while good examples of going back and forth between qualitative and quantitative research strands have been described in the literature (Greene, 2015; Hesse-Biber, 2010, 2012; Maxwell et al., 2015), accurate descriptions of the corresponding changes in ontology and epistemology are missing. Thus, mixed methods research practice has not yet provided evidence for this switch between different ontologies and epistemologies.

Critical Realism

A second paradigm for mixed methods research stems from critical realism, which assumes that there is one objective reality that can be known in various ways (Campbell, 1988). Much like the postpositivist view within the dialect-stance paradigm, the aim of critical realist mixed methods research is to obtain knowledge of a reality that is taken to exist in an independent, objective way. Contrary to the postpositivist view, knowledge in the critical realist paradigm is always provisional (Popper, 1963) and can be very diverse in nature, with reality capable of being known in many different ways. Critical realism emphasizes that reality includes experiences and feelings about which scientific knowledge can be gained.

A critical realist paradigm for mixed methods research has been developed by Joseph Maxwell (Maxwell, 2012; Maxwell & Mittapalli, 2010). In the words of Maxwell (2012),

Critical realists thus retain an ontological realism (there is a real world that exists independently of our perceptions, theories, and constructions) while accepting a form of epistemological constructivism and relativism (our understanding of this world is inevitably a construction from our own perspectives and standpoint). (p. 5)

Critical realism solves a number of problems of the dialectic-stance paradigm. Different from the dialectic-stance paradigm, the critical realist paradigm does not assume switching between distinct ontologies and epistemologies. In the critical realist paradigm, there is only one ontology and one epistemology, namely, that there is an objective reality that can be known in various ways. The utilization of multiple and mixed methods is legitimate, as quantitative and qualitative methods represent different ways of knowing. Thus, Lincoln’s (2010) problem of an improbable switch between deep-rooted beliefs has disappeared in the critical realist paradigm. Methods are no longer specifically linked to, or considered more appropriate for, a particular ontology or epistemology. In a critical realist paradigm, it is clear where to situate the outcomes of integration. Like everything else, integration takes place in one independently existing, objective reality, involving results that have been obtained via the various ways in which the objective reality can be known.

The critical realist paradigm, however, gives rise to new problems, including problems that are not encountered in the dialectic-stance paradigm. Like in all realist paradigms, research in the critical realist paradigm is guided by the notion of an objective reality. The researcher’s aim is to develop a better understanding of this objective reality. The following question then arises: How can it be determined if one inquiry has achieved a better understanding? If the notion of an objective reality is of any use, it should be possible to derive criteria from this objective reality from which researchers can judge whether a specific inquiry has led to a better, or perhaps a worse, understanding of reality. However, the problem is that researchers do not have direct access to this objective reality. Researchers do not possess a “God’s eye view” (Putnam, 1981) that shows them how reality “really” is, and therefore objective reality cannot be used to derive such criteria (Chakravartty, 2014). As a result, the notion of an objective reality has a problem of legitimacy. The question in turn is whether it makes sense to assume an objective reality: What can be gained by assuming an objective reality that should but cannot guide the conduct of research?

This argument has been rebutted by realists on the basis of various arguments. First, realists have stated that their position is not guided by objective reality but rather by the empirical success of the sciences. Given their success, our best theories must be true. Otherwise, the success of our theories would be a miracle (Chakravartty, 2014). Furthermore, given the successes of science, the hypothesis of scientific realism should be accepted as an accurate portrayal of the relationship between science and reality (Sankey, 2004). However, one could also take these arguments as simultaneously counterarguments showing the irrelevance of an objective reality to the daily business of science.

A second problem relates to the critical realist view on multiple realities. Among qualitative researchers, there is a widespread notion that there is not one world, but multiple subjective realities (e.g., Nicholls, 2017). Thus, the realities of a study’s participants may differ dramatically from the reality of the researcher. Furthermore, there may be differences among participants (Yin, 2016). This notion of multiple realities, however, is rejected by critical realism. In Maxwell’s (2012) words:

Thus, while critical realism rejects the idea of “multiple realities,” in the sense of independent and incommensurable worlds that are socially constructed by different individuals or societies, it is quite compatible with the idea that there are different valid perspectives on reality. (p. 9)

One argument against the critical realist position is that there are situations in which the notion of multiple realities makes sense. It makes sense to assume that participants in qualitative research to some extent live in different realities, with different concepts, habits, interests, and values, rather than claiming that they look differently at one objective reality. Assuming such differences especially makes sense if the aim of an inquiry is to describe these different realities. What would such a description gain from an “additional” objective reality? This does not mean that participants always live in different realities, nor does it mean that multiple realities are assumed in all qualitative research (Yin, 2016). It just means that, in certain situations, the assumption of different realities may be a fruitful standpoint.

Pragmatism

In the pragmatist paradigm (Morgan, 2007, 2014), the notion of various ontologies and epistemologies is abandoned and replaced by inquiry into the nature and consequences of human actions in a social context. The aim of research in the pragmatist paradigm is not to obtain timeless knowledge, but rather to solve temporary problems, understanding research as involving a succession of experiences where previous beliefs are revised based on specific research actions. During the conduct of research, pragmatist researchers regularly have to switch from their research outcomes to their current beliefs, to see whether the former match the latter. If the outcomes do not match the researcher’s current beliefs, these beliefs have to be updated. These updated beliefs, then, become the new “current beliefs,” and they form the basis for a new round of data analysis, or a new experiment. This process is repeated until the temporary problem has been solved. The pragmatist process of updating beliefs based on research outcomes aligns quite naturally with how the aim of conducting research is commonly viewed. In many if not all paradigms, the outcomes of research inquiry are used to modify existing “beliefs” (pragmatism), “intersubjective understanding” (constructivism), or “objective knowledge” (postpositivism).

In contrast to the dialectic stance and critical realism, pragmatism offers a fundamental criterion that can be used to determine whether or not research has been successful. In the pragmatist paradigm, progress can be defined as progress made in solving the temporary problem. Research can be said to be successful to the extent that it contributes to solving the temporary problem. Consequently, the critical realist problem—that research should be guided by something that is unable to guide it (the objective reality)—vanishes in the pragmatist paradigm. In the pragmatist paradigm, research is guided by the problem that it is attempting to solve. In this regard, it is quite possible to identify criteria for evaluating the extent to which an inquiry contributes to solving the problem. In determining criteria for validity based on a research problem, the pragmatist paradigm, unlike the critical realist paradigm, does not refer to an objective reality.

In the pragmatist paradigm, methodologies are not seen as separate entities that should be kept apart in their own research strands. Instead, research approaches are seen as relatively flexible configurations, and one research approach may combine elements of one methodology with elements of another methodology. Whether specific combinations of research elements are allowed in research is decided in the pragmatist paradigm at the level of the research community, which is assumed to hold shared beliefs about valid starting points of inquiry and legitimate research actions (Morgan, 2014). Elements from various methodologies can be combined as long as their combination forms part of the community’s shared beliefs. Thus, while researchers in the dialectic-stance paradigm switch between distinct methodologies, such a practice is abandoned in the pragmatist paradigm.

The pragmatist paradigm, however, has its own unique issues. First, following Lincoln’s (2010) notion that ontological and epistemological views are deep-rooted stances, one would expect that, by now, various mixed methods research communities would have been identified on the basis of their deep-rooted beliefs about which specific combinations of elements from positivist and constructivist methodologies are allowed. That this has not happened yet casts doubt on the idea that a research community is characterized by a set of shared beliefs.

Second, the pragmatist notion of combining elements from different methodologies is problematic. It is not clear how one could combine a research element that assumes that there is one reality that can be known objectively with a research element that assumes that there are multiple constructed realities. One cannot believe at the same time that there is both one reality and multiple realities. These beliefs are inconsistent and a set of shared beliefs that contains both is not trustworthy, unless these beliefs are interpreted in a special way. This does not contradict the both/and principle of dialectical pluralism, which states that mixed methods researchers should expect, accept, and work with differences in virtually every realm of inquiry, including reality (Johnson, 2017; Johnson & Schoonenboom, 2016). According to the both/and principle, a researcher accepts and works with differences, for example, works with researchers who have different ontologies and epistemologies. The message of this paragraph is different, stating that a researcher as a person or a research community as an entity cannot hold inconsistent beliefs, such as that there is both one reality and multiple realities.

Third, in addition to the issue that no mixed methods research communities have been defined based on their shared beliefs, the notion of shared beliefs itself is problematic. The notion of shared beliefs as constitutional of a community, including communities of mixed methods researchers, has been attacked most passionately and successfully by Joseph Maxwell (Maxwell, 2012; Maxwell & Mittapalli, 2010). Culture is not shared, but it is participated in (Aberle, 1960), and sharing is only one possible form of participation (Maxwell, 2012). Rather than by shared goals, a culture is characterized by diversity. There is no evidence of a functional necessity of shared goals, motives, or beliefs (Wallace, 1970). This makes shared beliefs within a research community as the starting point of inquiry and legitimate research actions problematic.

Last, with the pragmatist disappearance of ontologies and epistemologies, the idea of multiple realities disappears as well. Consequently, unlike constructivism, pragmatism cannot accommodate this common and useful starting point of qualitative inquiry.

The Performative Paradigm

A Performative Ontology and Epistemology

Like pragmatism, the performative paradigm focuses on research actions. Unlike pragmatism, however, it assumes a specific, constructivist ontology and epistemology. The performative paradigm assumes the existence of multiple realities that can be known and investigated in various ways. It agrees with the idea of multiple realities, the idea that people to some extent live in different realities, with different concepts, habits, interests, and values, rather than claiming that they look differently at a single objective reality. These different realities are dynamic and changeable. Moreover, the concepts researchers use to speak about these realities are not fixed, but change as these realities change. Realities are thus multiple, varied, and changing. This ontology and epistemology are borrowed from dialectical pluralism (Johnson, 2017).

The performative paradigm solves a number of the problems of the previous paradigms. In the performative paradigm, mixed methods researchers do not switch between ontologies, epistemologies, or research stances. Rather, they stay and live in multiple realities that can be known in various ways. This lack of a switch corresponds to the experiences of Bryman’s (2007) mixed methods researchers and to Lincoln’s (2010) argument that a researcher does not switch stances.

While the performative paradigm, with its focus on temporality, comes close in nature to the pragmatist paradigm, it differs from pragmatism in that different worlds, and research as an attempt to better understand the laws and habits of worlds, play an important role. Whereas in the pragmatist paradigm, the outside world is formed by the experiences of researchers and their social context, the performative paradigm assumes the existence of distinct worlds. As opposed to the external, objective reality of critical realism, the distinct worlds of the performative paradigm are viewed as temporary and created. In the performative paradigm, worlds are not objectively given, but arise from the concepts that researchers use to speak about them in a specific, time-bound context. These concepts are viewed as constituting concepts, as they constitute worlds. The performative paradigm assumes that researchers’ concepts are intersubjective in the sense that they are sharable. In line with Maxwell’s argument, this does not mean that constituting concepts are shared by all members of a specific community, nor does it mean that all people who know a specific concept have a shared definition of it. Rather, it means that researchers can share a concept and develop criteria for judging whether someone understands a concept (e.g., if a person uses it in a correct way, is able to explain the concept, or is able to provide examples; Wittgenstein, 1953).

Precisely because concepts can be used in various ways, it is often necessary that researchers explain what they mean by a specific concept in a specific context. Putnam (1987) provides an example in which the answer to the question how many objects exist in a given world might differ depending on the definition of what an object is. For purposes of demonstration, Putnam’s example is simplified. Assume that there is a world which contains the objects A, B, C, A, C, and C. A researcher may ask, “How many objects exist in this world?” This question has two interpretations, depending on what is meant by object. In this example, the researcher may interpret object to mean “different types of objects,’’ in which case the answer to the question is three, namely, A, B, and C. Using terminology borrowed from linguistics, this world contains three different object types. Alternatively, the researcher may interpret object as referring to each separate instantiation of an object type, or in linguistic terms, to each separate token. In that case, the answer would be six, namely, A, B, C, A, C, and C. Under this interpretation, this world contains six token objects.

This means that, to avoid confusion, a researcher should make clear whether he or she refers to type objects or token objects. In that case, according to Putnam (1987),

Once we make clear how we are using “object” (or “exist”), the question “How many objects exist?” has an answer that is not at all a matter of “convention.” That is why I say that this sort of example does not support radical cultural relativism. Our concepts may be culturally relative, but it does not follow that the truth or falsity of everything we say using those concepts is simply “decided” by the culture. (p. 174)

Putnam’s quotation brings to the fore two differences between constituting concepts and statements in which constituting concepts are used. First, the quotation shows that “truth” is a qualification that does not apply to constituting concepts, but may apply to statements in which the constituting concepts are used. Researchers can choose how they use the constituting concept object (type object or token object), with one choice being no less true than the other, although it can be less or more conventional. In contrast, the qualification “truth” can be applied to the statement This world contains six objects, and the result depends on the researcher’s definition of the term object (true for token objects, false for type objects).

Second, and related, Putnam’s quotation shows that while constituting concepts are intersubjective in the sense of sharable, statements in which these constituting concepts are used contain an element of objectivity. Researchers do not have the freedom to define the truth of a statement separate from their definition of its constituting concepts. When a researcher has chosen to interpret object in the example world as “token object,” then he or she cannot define the number of objects separate from this choice (as, e.g., three). Given a definition of object as “token object,”it follows that the number of objects in this world is six. The true statement The number of objects in this world is six, when referring to token objects, is in a specific sense objectively true. Putnam, thus, introduces a concept of truth that does not depend on an independent reality out there, but on a clear explanation of what the constituting concepts mean in utterances that are considered true or false. As a consequence, truth is bound to a specific, historical context.

Putnam’s example can be viewed as an investigation into a very simple world with inhabitants A, B, C, A, C, and C, where the right answer is immediately clear once the researcher defines the terms object and exist. In contrast, scientific research investigates very complex worlds, based on concepts that interact with each other in endlessly complex ways. In such complex worlds, answers to research questions cannot be easily derived directly from a definition of its constituting concepts and data about the world’s inhabitants. Instead, scientific research uses very divergent and often complex research methods from which researchers can discover properties of these worlds that were previously unknown. Thus, while researchers define their worlds by their constituting concepts, they then discover these worlds’ often hidden properties.

The foundational elements of the performative paradigm can be explained using the example of neural networks. The advent of the computer enabled the development of neural networks, which consist of simple nodes that can be either on or off and which are connected to each other through adaptable weights. A network can be given an input pattern and a corresponding output pattern. In going through numerous cycles, the neural network adapts its weights using a prespecified algorithm so that it is able in the end to generate the correct output when provided with a corresponding input. Neural networks are so complex that researchers cannot directly and easily answer questions about their characteristics and output patterns based on their constituting concepts and algorithms. Only through scientific inquiry can researchers discover specific characteristics of specific types of neural networks. In neural network inquiry, simulations play an important role.

This neural network example speaks to the fact that constituting concepts can emerge, be discovered, or be developed at a specific moment in history. In addition, it shows that, in using these concepts, worlds can be created, in this case neural networks. It shows that characteristics of specific neural networks cannot be defined separate from anything else, but rather follow from their constituting concepts. Finally, the example shows that to discover the characteristics of neural networks, complex scientific research is needed, in this case research that makes use of complex simulations.

The characteristics of worlds that result from scientific research are not eternally valid and may change when these worlds or their constituting concepts change. However, based on both a definition of what the constituting concepts mean and an understanding of how researchers can investigate the interactions between these concepts, the discovered properties of a world can be considered objectively valid for those who accept the definitions of the constituting concepts as well as the method of investigation. Because concepts and methods may vary from place to place and change over time, objectivity is always bound to a certain time and context.

How the Performative Research Process Leads to Objectivity

Striving for objectivity is an integral goal of the performative paradigm. Before the role of objectivity is defined and explained, the question is raised how objectivity is obtained within the performative paradigm. The question is how objectivity can be obtained if not via a reality check (critical realism), an intersubjective understanding (constructivism), or through the revision of researchers’ beliefs and solving problems (pragmatism). The answer to this question is provided by Pickering’s performative view of research (Pickering, 1995). Pickering (1995) describes the conduct of research as a dance of agency, which is characterized by an alternation of actions performed by the researcher and actions performed by the world created by the researcher. To continue this metaphor of the dance, each movement within a dance of agency is characterized by three steps. In each step, as in a real dance, either the researcher takes the lead, and the world follows, or the world takes the lead and the researcher follows. In those steps in which the researcher takes the lead, the researcher is active, and imposes something on the world. In those steps in which the researcher follows, the researcher steps back and observes how the world reacts to what had been imposed.

In the first step, the researcher takes an active role. The researcher starts with particular concepts that he or she uses to create a particular world, as well as with particular ideas about how these concepts interact in the created world. Pickering refers to this process as goal setting and modeling: The researcher’s ideas are imposed on the world. In the second step, the researcher takes a passive role. By observing the outcomes of imposing his or her ideas on the world, the researcher learns whether his or her ideas about the interactions between the concepts are correct or should be revised. Pickering refers to this as the resistance of material agency. In the third step, the researcher again assumes an active role. On the basis of what the researcher has learned about his or her initial ideas from observing the world, the researcher then adapts his or her ideas or model of the world to accommodate the world’s feedback. Pickering calls this the accommodation of human agency. The feedback cycle as a whole is called a mangle of practice, because it “mangles” the researcher’s ideas. Subsequently, a new feedback cycle starts, and alternating cycles continue until what Pickering refers to as interactive stabilization is reached.

The dance of agency depicts objectivity as emergent, “posthuman,” and the result of a feedback dialogue. In Pickering’s (1995) words,

the detachment from the intentional structure of human agency through encounters with material and disciplinary agency (themselves proper to no individual subject) is the basis sense of the objectivity of science that the mangle makes available to us. (p. 195)

These characteristics of Pickering’s approach can be demonstrated using a real-life example from mixed methods research in education (Glewwe, Kremer, & Moulin, 2009). Education has a significant impact on people’s well-being and income. In many developing countries, however, education suffers from a lack of adequate organization and resources. Thus, educational economists have inquired as to what measures are most effective and cost-efficient to improve education in developing countries. In this regard, there is a large consensus among economists that providing textbooks to schools where they are scarce can substantially increase test scores. When Glewwe et al. (2009) began their inquiry into the effect of providing textbooks to primary school children in rural Kenya, they did so with the belief that providing textbooks would positively affect students’ quantitative test scores. However, their intervention study, wherein schools were randomized to treatment, did not find the expected effect. Their follow-up qualitative inquiry in the schools, in which children were asked to read the textbooks, revealed that the majority of the children were unable to read these difficult textbooks, which were written in English (not their first language). Their subsequent quantitative subgroup analysis showed that the textbooks had an effect for excellent students, who were able to read their textbooks.

Table 1 displays the subsequent research stages of the Glewwe et al. (2009) example. The models involved are relatively simple and consist of hypotheses. Each research stage starts with an initial hypothesis (Column 1). This hypothesis is then put through a mangle (Column 2), in the example consisting of a qualitative or a quantitative analysis of empirical data. While all other columns refer to the actions of the researcher, Column 3 shows what the material world of a specific mangle tells the researcher. Based on the resistance of this world to the model, the researchers accommodate their hypothesis accordingly (Column 4). Note that the analysis of qualitative observations has bearing on two different hypotheses, shown as separate rows in Table 1. First, the analysis of the qualitative observations provides an explanation for the no-effect finding that resulted from the quantitative overall analysis. Second, the analysis of the qualitative observations makes one of the assumptions of the first hypothesis explicit, namely that providing textbooks can only have an effect for children who are able to read these books. This assumption is then used to differentiate the first hypothesis into separate hypotheses for children who can read their textbook and children who cannot read their textbook.

Table 1.

The Mangle of Practice Applied to Glewwe et al. (2009).

Initial hypothesis	Mangle	Feedback of the mangle’s world	Hypothesis updated by the researchers to accommodate the world’s feedback
[Not provided]	Previous research	[Not provided]	Providing textbooks where they are scarce has a positive effect on primary school students’ quantitative test scores.
Providing textbooks where they are scarce has a positive effect on primary school students’ quantitative test scores.	Quantitative overall analysis	Point to an exception	Providing textbooks has no effect on the test scores of pupils within primary schools in rural Kenya.
Providing textbooks has no effect on the test scores of pupils within primary schools in rural Kenya.	Analysis of qualitative observations	Explains exception	The majority of the children were unable to read their textbooks; for these children, the textbook could not have had an effect.
Providing textbooks where they are scarce has a positive effect on primary school students’ quantitative test scores.	Analysis of qualitative observations	Makes explicit an implicit assumption	Providing textbooks can only have an effect for children who are able to read the books.
Providing textbooks where they are scarce has a positive effect on primary school students’ quantitative test scores.	Analysis of qualitative observations	Leads to a differentiated hypothesis	Providing textbooks has a positive effect on the test scores of children who can read the textbook, and no effect for children who cannot read the textbook.
Providing textbooks has a positive effect on the test scores of children who can read the textbook, and no effect for children who cannot read the textbook	Quantitative subgroup analysis	Confirms differentiated hypothesis	Providing textbooks to pupils within primary schools in rural Kenya has an effect for excellent students, who are able to read their textbooks, and no effect for students who cannot read the textbooks.

This performative research procedure has much in common with the pragmatist research procedure. However, the focuses of the two are different. Pragmatism assumes that what one finds in reality does not necessarily correspond to one’s prior beliefs, while the performative paradigm emphasizes the creation of a new world by the researcher with as-yet-unknown properties. A pragmatist approach is characterized by an alternation between a researcher’s beliefs and his or her actions. The performative paradigm is characterized by an alternation between the researcher and the world created by the researcher, the properties of which by far surpass that which the researcher may have been able to imagine in advance.

Objectivity according to Pickering (1995) originates from the feedback that a researcher receives from the world concerning the interactions between the constituting concepts and from the subsequent accommodations of the researcher’s ideas about these interactions. The more cycles of feedback that have been completed, the more objective the result. Objectivity is always a matter of degree and therefore complete objectivity cannot be achieved. This is not so, as in critical realism, because the world cannot be known objectively, but rather because every world is temporary and always “open” to some degree, and because it is always possible for researchers to develop a new test of objectivity.

In the performative paradigm, objectivity may also vanish. When the constituting concepts of a world change, the world itself changes, potentially up to a point where it becomes unrecognizable. Because of this change in constituting concepts, statements referring to these concepts may no longer be true, as Putnam’s example has demonstrated.

As objectivity is a matter of degree and originates from the relationship between a researcher and his or her created worlds, objectivity can and does already arise in a situation that involves only one researcher. Acceptance by peer researchers, although it contributes to objectivity, is not required to obtain a first stage of objectivity. For example, when the categories derived in a grounded theory analysis prove not to fit the data and the researcher consequently adapts the categories, the revised categories can be said to be more objective than the originally derived categories. That is, the categories have passed their first reality test; they have survived their first mangle of practice.

A Performative Paradigm for Mixed Methods Research

The performative paradigm is based on Pickering’s view of objectivity as temporary, emergent, “posthuman,” and the result of a feedback dialogue. Striving for objectivity is seen as one main reason for conducting research. Interaction with differences, important in the dialectic stance (Greene, 2007; Johnson, 2017), is what happens in the mangles of practice.

The mangle of practice, thus, can provide a foundation for mixed methods research. Mangles can be of different natures and this includes qualitative and quantitative natures. For example, qualitative grounded theory is a mangle, but so is a quantitative statistical test or a peer review. The more mangles a researcher applies, and the more varied these mangles are, the more objective the result becomes. A higher degree of objectivity can be obtained through mixed methods research in two ways. First, researchers may obtain an ever-increasing degree of objectivity by subsequently applying additional and different mangles to the results of previous mangles. This is the well-known sequential design (Schoonenboom & Johnson, 2017), in which data collection and data analysis of one method take place after data collection and data analysis of the other method (Creswell & Plano Clark, 2011). Second, a conclusion will be more objective when it is the result of applying two different mangles at the same time (e.g., applying two different research methods to answer the same research question), which is known as a concurrent design (Creswell & Plano Clark, 2011).

In the performative paradigm, rather than being able to switch between different ontologies and epistemologies, a mixed methods researcher is described as someone who accepts different kinds of objectivity as legitimate and who accepts that different methods may be used to obtain a higher degree of objectivity. In practice, however, one reason why some quantitative and qualitative researchers do not perform mixed methods research is that they reject the other approach. I believe that such rejections are more related to a researcher’s understanding of the results obtained using the other approach than to a researcher’s understanding of reality. I believe that a quantitative researcher’s rejection of qualitative research results often does not stem from a rejection of the idea of multiple realities, but rather from the belief that the results of qualitative research are not objective (e.g., because the sample was not representative or large enough to generalize results to a different population). While these objections are warranted for quantitative research, they do not apply to qualitative research, which aims at understanding rather than at generalization. A good mixed methods researcher is someone who accepts that quantitative (i.e., generalization) and qualitative aims (i.e., understanding) are both legitimate aims for research, and that both quantitative and qualitative research can lead to a higher degree of objectivity.

A summary of all four paradigms (i.e., the dialectic stance, critical realism, pragmatism, and the performative paradigm) can be found in Table 2.

Table 2.

A Descriptive Summary of the Four Paradigms for Mixed Methods Research.

	The dialectic stance example	Critical realism	Pragmatism	Performative paradigm
Description	Two views on reality and how reality can be known.	There is one objective reality, which can be known in various ways.	Consequences of actions in a social situation	Exploration of unknown created worlds
Ontology and epistemology	Two distinct views: one view of an objective reality that can be known objectively and one view of multiple realities that can be known subjectively.	There is one objective reality, which can be known in various ways.	The idea of an eternal ontology and epistemology is replaced by the interaction of people’s beliefs and actions, both of which are temporary, in experiences.	Reality is plural and multiple, and can be known in multiple ways.
The essence of . . .	Objective/intersubjective knowledge	Subjective knowledge	Human experience	Knowledge development
Knowledge is . . .	Timeless	Provisional	Replaced by action	Temporary
In research . . .	Different views of reality are kept apart in different research strands.	There is only one view of reality, namely, one objective reality, which can be known in various ways.	A research approach may combine elements of one view of reality with elements from a different view of reality.	There is only one view of reality, namely, reality is plural and multiple, and can be known in multiple ways.
The researcher’s task	To describe reality/realities.	To describe reality.	To solve problems through the interaction of beliefs and actions in experiences.	To create and subsequently explore new worlds.
The research process	The research process depends on which of the two distinct research strands the researcher is working within.	The research process can proceed in various ways because the world can be known in various ways.	Beliefs that have become problematic are examined and resolved through action.	A continual switch between the ideas of the researcher and the reality of the data.
Inquiry results in . . .	Objective/intersubjective knowledge	A better yet incomplete understanding of reality	A contribution to solving a problem	A better understanding of a created world
What determines improved understanding?	Better correspondence to reality/intersubjective discussion	Better correspondence to reality	A solution to a problem	Passing through successive mangles
A bridge between constructivism and postpositivism is formed by . . .	The researcher, who switches between both views	The world itself is objective, hence, the use of “objective” methods. The world can be known in various ways, hence, the use of “subjective” methods.	Shared beliefs within a community	A succession of different kinds of mangles
What should the mixed methods researcher accept?	The existence of and switching between two distinct views	That there is one objective reality, which can be known in various ways	The shared beliefs of their research community	That obtaining objectivity is a process to which many types of mangles can contribute
How does one connect qualitative and quantitative research?	Remains unclear	Remains unclear	Beliefs in a community and appropriate actions contain both quantitative and quantitative research elements	Methods build on the results obtained by a previous method; results and methods can be of various kinds.

Discussion

In this article, I have presented a performative paradigm for mixed methods research. The ontology and epistemology of the performative paradigm stem from dialectical pluralism (Johnson, 2017), which holds that multiple realities can be known in various ways. In addition, realities in the form of worlds are temporary and emergent (Pickering, 1995). In performative research inquiry, a researcher is viewed as someone who creates new, unknown worlds, which are then subjected to further investigation. Across and within research inquiries, an ever-higher degree of objectivity is obtained in successive “mangles,” rounds of feedback, in which the researcher listens to the created world’s feedback and adapts his or her research models accordingly. This objectivity, however, comes to an end when the constituting concepts of a given world change. This leads to a change of the entire world and affects the truth of statements about the world, as Putnam’s example has shown. Research involves a succession of multiple mangles. These mangles can be of different kinds, which allows room for both quantitative and qualitative inquiry. Comparing a researcher’s categories to what interviewees have said and his or her subsequent accommodation of these categories (such as in grounded theory) is as much a process of resistance and accommodation as is utilizing quantitative data to test a statistical model. A mixed methods researcher is someone who recognizes both forms of mangling as legitimate.

In the performative paradigm, research is viewed as a dialogue: a dialogue between the researcher, the material world, and the social world. The question arises, what role dialogue plays in the other three paradigm, and how this role differs between paradigms. Answering this question reveals the opportunities and at the same time the limitations of the performative paradigm. The dialectic stance emphasizes dialogue among researchers with different worldviews or mental models. This includes dialogue between researchers who hold that there is one reality and researchers who hold that there are multiple realities. This last type of dialogue, unlike others, is not accommodated within the performative paradigm, which assumes that there are multiple realities, and thus does not provide room for the idea that there is only one reality. While this last type of dialogue is clearly not impossible, this is a limitation of the performative paradigm. Thus, while a mixing of paradigms in the sense of worldviews (Greene, 2007) is not impossible within the performative paradigm, there are some restrictions.

Within critical realism, there is a dialogue between researcher and reality. This is true for the performative paradigm as well, the difference being that reality is assumed to be uniform and objective in the critical realist paradigm, whereas objectivity is only temporary and emergent in the performative paradigm, and each reality is only one of all possible realities. Finally, in the pragmatic paradigm, the dialogue takes place between the researcher’s previous beliefs, the research actions, and the temporary problems. The performative paradigm recognizes all three elements but deemphasizes the role of the temporary problem. It is not denied that a wish to solve a particular problem may provide a good reason for conducting research. However, once researchers enter into their created worlds and start to interact with them, they may discover issues that are unrelated to their original problem. Therefore, the extent to which the temporary problem is solved is not a criterion for good research or for objectivity in the performative paradigm. Again, this is a limitation of the performative paradigm.

This article has methodological significance, as it demonstrates the importance of reflecting on common images from everyday research practice that have an intuitive appeal (Campbell, 1978; Wittgenstein, 1953). In this article, the following four images of research practice have been used to criticize existing paradigms and develop the performative paradigm:

Mixed methods researchers do not experience a continual switch between different ontologies and epistemologies (Bryman, 2007).

During an inquiry, researchers do not switch between deep-rooted stances about reality (Lincoln, 2010).

People partly live in different worlds rather than having different views on one and the same reality.

As a researcher applies his or her research models to reality, reality talks back.

The first two images show that any paradigm that presupposes that researchers switch between different ontologies and epistemologies during the conduct of research has a problem. Researcher do not experience such a switch (Image 1), and it is unlikely that they switch, because ontologies and epistemologies are deep-rooted stances that do not change easily (Image 2). Hence, the dialectic-stance paradigm, which assumes such a switch, has a fundamental problem. The third image shows that any paradigm that does not include the concept of multiple worlds has a problem. It makes sense to say, and it is a recognized stance in the literature on qualitative research, that people to a certain extent live in different worlds (Image 3). Hence, the critical realist paradigm has a fundamental problem, because it assumes the existence of one objective world. The fourth image shows that a paradigm ideally should give agency to the material world. Researchers experience that reality talks back (Image 4). The performative paradigm honors this experience by including feedback given by the world as an element of the research dance of agency. One aim of this article was to show that it makes sense to take these four images seriously and give them a proper place in the foundations of mixed methods research. Only when this is done, the mixed methods community is able to overcome confused ideas such as switching between distinct ontologies and epistemologies, or the existence of an objective world that is unable to guide research, and replace them with the more plausible image of a researcher who creates multiple research worlds, interrogates them, and listens to them.

Footnotes

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) received no financial support for the research, authorship, and/or publication of this article.

References

Aberle

D. F.

(1960). The influence of linguistics on early culture and personality theory. In Dole

G. E.

Carneiro

R. L.

(Eds.), Essays in the science of culture in honor of Leslie A. White (pp. 1-29). New York, NY: Thomas Y. Crowell.

Bazeley

(2012). Integrative analysis strategies for mixed data sources. American Behavioral Scientist, 56, 814-828. doi:10.1177/0002764211426330

Bazeley

Kemp

(2012). Mosaics, triangles, and DNA: Metaphors for integrated analysis in mixed methods research. Journal of Mixed Methods Research, 6, 55-72. doi:10.1177/1558689811419514

Bryman

(2007). Barriers to integrating quantitative and qualitative research. Journal of Mixed Methods Research, 1(1), 8-22. doi:10.1177/2345678906290531

Butler

(1993). Bodies that matter. On the discursive limits of sex. London, UK: Routledge.

Campbell

D. T.

(1978). Qualitative knowing in action research. In Brenner

Marsh

Brenner

(Eds.), The social contexts of method (pp. 184-209). New York, NY: St. Martin’s Press.

Campbell

D. T.

(1988). Methodology and epistemology for social sciences: Selected papers. Chicago, IL: University of Chicago Press.

Chakravartty

(2014). Scientific realism. In Zalta

E. N.

(Ed.), The Stanford encyclopedia of philosophy (Spring 2014 ed.). Stanford, CA: Stanford University Press. Retrieved from http://plato.stanford.edu/archives/spr2014/entries/scientific-realism/

Creswell

J. W.

Plano Clark

V. L.

(2011). Designing and conducting mixed methods research. Thousand Oaks, CA: Sage.

10.

Fetters

M. D.

Curry

L. A.

Creswell

J. W.

(2013). Achieving integration in mixed methods designs—Principles and practices. Health Services Research, 48, 2134-2156. doi:10.1111/1475-6773.12117

11.

Glewwe

Kremer

Moulin

(2009). Many children left behind? Textbooks and test scores in Kenya. American Economic Journal: Applied Economics, 1(1), 112-135. doi:10.1257/app.1.1.112

12.

Greene

J. C.

(2007). Mixed methods in social inquiry. San Francisco, CA: Jossey-Bass.

13.

Greene

J. C.

(2015). Preserving distinctions within the multimethod and mixed methods research merger. In Hesse-Biber

Johnson

R. B.

(Eds.), The Oxford handbook of multimethod and mixed methods research inquiry (pp. 606-615). New York, NY: Oxford University Press.

14.

Hesse-Biber

(2010). Qualitative approaches to mixed methods practice. Qualitative Inquiry, 16, 455-468. doi:10.1177/1077800410364611

15.

Hesse-Biber

(2012). Weaving a multimethodology and mixed methods praxis into randomized control trials to enhance credibility. Qualitative Inquiry, 18, 876-889. doi:10.1177/1077800412456964

16.

Johnson

R. B.

(2017). Dialectical pluralism: A metaparadigm whose time has come. Journal of Mixed Methods Research, 11, 156-173. doi:10.1177/1558689815607692

17.

Johnson

R. B.

Schoonenboom

(2016). Adding qualitative and mixed methods research to health intervention studies: Interacting with differences. Qualitative Health Research, 26, 587-602. doi:10.1177/1049732315617479

18.

Lincoln

Y. S.

(2010). “What a long, strange trip it’s been …”: Twenty-five years of qualitative and new paradigm research. Qualitative Inquiry, 16, 3-9. doi:10.1177/1077800409349754

19.

Maxwell

J. A.

(2012). A realist approach for qualitative research. Thousand Oaks, CA: Sage.

20.

Maxwell

J. A.

Chmiel

Rogers

(2015). Designing integration in mixed method and multi-method research. In Hesse-Biber

Johnson

(Eds.), The Oxford handbook of multimethod and mixed methods research (pp. 223-239). New York, NY: Oxford University Press.

21.

Maxwell

J. A.

Mittapalli

(2010). Realism as a stance for mixed methods research. In Tashakkori

Teddlie

(Eds.), Sage handbook of mixed methods in social & behavioral research (2nd ed., pp. 145-167). Thousand Oaks, CA: Sage.

22.

Morgan

D. L.

(2007). Paradigms lost and pragmatism regained: Methodological implications of combining qualitative and quantitative methods. Journal of Mixed Methods Research, 1, 48-76. doi:10.1177/2345678906292462

23.

Morgan

D. L.

(2014). Pragmatism as a paradigm for social research. Qualitative Inquiry, 20, 1045-1053. doi:10.1177/1077800413513733

24.

Nicholls

(2017). Qualitative research. Part 1: Philosophies. International Journal of Therapy and Rehabilitation, 24(1), 26-33. doi:10.12968/ijtr.2017.24.1.26

25.

O’Cathain

Murphy

Nicholl

(2010). Three techniques for integrating data in mixed methods studies. British Medical Journal, 341, c4587. doi:10.1136/bmj.c4587

26.

Pickering

(1995). The mangle of practice: Time, agency, and science. Chicago, IL: University of Chicago Press.

27.

Popper

(1963). Conjectures and refutations: The growth of scientific knowledge. London, UK: Routledge.

28.

Putnam

(1981). Reason, truth and history. Cambridge, UK: Cambridge University Press.

29.

Putnam

(1987). The many faces of realism. Chicago, IL: Open Court.

30.

Sankey

(2004). Scientific realism and the God’s eye point of view. Epistemologia, 27, 211-226.

31.

Schoonenboom

Johnson

R. B.

(2017). How to construct a mixed methods research design. Kölner Zeitschrift für Soziologie und Sozialpsychologie [Cologne Journal for Sociology and Social Sociology], 69(Suppl. 2), 107-131. doi:10.1007/s11577-017-0454-1

32.

Wallace

A. F. C.

(1970). Culture and personality (2nd ed.). New York, NY: Random House.

33.

Wittgenstein

(1953). Philosophical investigations ( Anscombe

G. E. M.

, Trans.). Oxford, UK: Basil Blackwell.

34.

Yin

R. K.

(2016). Qualitative research from start to finish (2nd ed.). New York, NY: Guilford Press.