The natural sciences in the information systems research literature: Correcting misinterpretations

In their paper entitled “Demystifying Beliefs about the Natural Sciences in IS,” the authors claim that many beliefs that IS scholars have held for or against the natural sciences are erroneous or misleading. They say that, in the information systems (IS) research literature, natural science methods have been characterized as using quantitative research methods, producing objective knowledge, with the overall goal of finding deterministic general laws. The authors state that these IS beliefs are wrong. In reality, there is great variety among natural science methods. The authors show that the natural sciences use qualitative as well as quantitative methods; the data they collect are subjective (not objective or value-free); and they rarely if at all find deterministic general laws.

I agree with the second part of their argument. That is, I agree that there is great variety among the natural sciences. I also agree that the natural sciences use qualitative as well as quantitative data, that the data they collect are subject to interpretation, and that they rarely find deterministic laws. However, I disagree with their characterization of IS views about the natural sciences. They claim to be demystifying beliefs about the natural sciences in IS, but I believe they themselves create more confusion by misinterpreting what earlier IS scholars said. They misinterpret much of the IS research literature and miss the main point that many IS authors were trying to make. They are criticizing a straw man.

Let me start by quoting the authors themselves:

. . . natural science theorizing commonly involves crafting various different idealized models for various purposes, which often contain deliberate falsehoods or misrepresentations . . . In such cases, the model should not be assumed to represent the truth. Natural scientists seem to sacrifice the assumed truth for achieving some other (higher) purpose . . .

I agree with the authors that natural scientists craft “various idealized models for various purposes.” A point that the authors fail to note, however, is that these idealized models are usually taught to students as the truth. These models appear in textbooks and they are used as exemplars for much scientific research. Let me illustrate.

Randomized control trials (RCTs) are often described as the “gold standard” in pharmaceutical and medical research. As Bothwell et al. (2016) describe,

By the turn of the 21st century, RCTs had achieved the status of gold standard for therapeutic evidence—but one with well-documented limitations. Physicians continue to pursue alternative methods of knowledge production that are faster or less expensive than RCTs, or that claim to answer questions that RCTs cannot. Yet beyond medicine, RCTs are increasingly emulated, even idealized.

Why did RCTs become idealized as the gold standard in scientific research? The reason is, as Bothwell et al. (2016) explain, because RCTs are supposed to reduce bias and enhance the accuracy of clinical experimentation. The quantitative data obtained from RCTs are analyzed statistically in order to distinguish significant from insignificant results. Hence, while Bothwell et al. (2016), along with many other scientists, are well aware of the limitations of RCTs, they are still “emulated, even idealized” in medicine and many other scientific fields.

Hence, while I agree with the authors that natural sciences do not just use quantitative methods, quantitative methods are still held up by natural scientists themselves as an idealized model—they are regarded as the “gold standard.” All scientists know that the real world is messy and that their models capture only part of the truth, but this is not what scientists teach as the ideal. It is the idealized model that has been critiqued in the IS research literature, not the actual practice of science.

I will now illustrate how the authors misrepresent the IS research literature with respect to IS scholars’ discussions about natural science. They misrepresent IS scholars by conflating what IS scholars say about an “idealized model” of natural science versus the actual practice of natural science. As the authors themselves admit, these two things are quite different.

The first example is from my own book where the authors say this:

. . . claiming that survey research is quantitative, or that “all quantitative scholars emphasize numbers more than anything else” (Myers, 2013: 7), also seems problematic. Survey designers most likely use qualitative judgment in various places. For example, if researchers contextualize survey instruments, then how is this contextualization process quantitative and not qualitative, and how can one say that it “emphasize[s] numbers more than anything else.” (Myers, 2013: 7)

Thus, the qualitative/quantitative dichotomy seems problematic. However, if the dichotomy must be used, then numerous investigative approaches in biology, biochemistry, and chemistry research may not necessarily entail statistical analysis or “emphasize numbers more than anything else.” (Myers, 2013: 7)

The authors claim here is that Myers characterizes all quantitative researchers and natural scientists as using quantitative data only. They portray Myers as arguing that survey designers and natural scientists ignore qualitative judgment. The implication is that Myers, like many other IS scholars, has a simplistic and erroneous view of research in the natural sciences.

The problem here is that this is a misinterpretation of the quotation. The authors are criticizing a straw man. First, notice that I never said that quantitative scholars ignore qualitative data—I just said that they emphasize numbers more than anything else. Numbers are the ideal, just as they are in RCTs, but this does not mean that scholars in the natural sciences or those conducting RCTs ignore everything else. I never said that. Numbers are the gold standard, but not the only source of evidence.

Second, just a few pages later in the same work I say this:

Bernstein reviews the commonly assumed differences between the natural and social sciences and argues that all of the epistemological assumptions which supposedly distinguish the human sciences apply equally well to the natural sciences. Bernstein points out that there is a necessary hermeneutical dimension to all science. Kuhn’s historical analysis of the nature of paradigm shifts in science supports this view . . . (Myers, 2013: 40)

Here, I am pointing out that all science, including the natural sciences, involves a necessary hermeneutical dimension” (i.e. interpretation). Hence, to claim that I am making a sharp distinction between the natural sciences and the social sciences is simply incorrect. The authors fail to see that I was describing an idealized model of quantitative research in the introductory section of the book. It was never intended as a description of natural science practice.

The second example of how the authors misrepresent earlier IS research is their discussion of the article by Orlikowski and Baroudi (1991). The authors say this about Orlikowski and Baroudi’s article:

Orlikowski and Baroudi (1991) associated positivist natural science research with the view that the researcher “does not intervene in the phenomenon of interest” (p. 9) . . . IS readers learn that natural science oriented researchers “[do] not intervene in the phenomenon of interest” (Orlikowski and Baroudi, 1991: 9). Natural science research can, however, intervene “in the phenomenon of interest.”

The authors thus claim to be correcting the mistaken views of scholars such as Orlikowski and Baroudi. Natural scientists can intervene in the research setting, something that Orlikowski and Baroudi apparently denied. The problem here is that this statement is also a misrepresentation of what Orlikowski and Baroundi actually said. The authors insert the words “natural science” to give the impression that Orlikowski and Baroudi are talking about natural science research. But they are not. The focus of the original article is about the epistemological assumptions used in IS research, not the natural sciences. While Orlikowski and Baroudi describe “positivism” as “a research tradition that has its roots in the natural sciences,” the natural sciences are not their focus. In fact, they themselves say that “Recent work in the post-empiricist philosophy of science (Bernstein, 1985; Bhaskar, 1978) has begun to question the validity of these assumptions for the practice of natural science” (Orlikowski and Baroudi, 1991: 9).

Hence, to claim that Orlikowski and Baroudi (1991) have a simplistic view of research in the natural sciences is incorrect. Again, they are criticizing a straw man. The original article was focusing on how positivist IS researchers have an idealized view of natural science research. To say that Orlikowski and Baruodi, like Myers, ignore the fact that natural scientists interpret their data, some of which is qualitative, or that natural scientists do not intervene in the research setting, is simply incorrect.

A third example showing how the authors misrepresent IS research is their discussion of an article by Lee (1989). The authors criticize Lee’s portrayal of “natural science model of social science,” saying that no IS scholar including Lee seems to have questioned “whether the prevailing beliefs about natural science in IS are supported by evidence from the contemporary philosophy of natural science or from the natural science practice.” The implication is that Lee along with many other IS scholars is unaware of natural science practice and that Lee takes his natural science model of research as fact.

This again, however, is a misrepresentation of what Lee actually said. Lee said this:

. . . there are numerous models of science from which to select. Indeed, philosophers of science—the scholars who make it their job to observe scientists and to explain what it is that scientists do—have not yet settled among themselves, on a single model of what science is. The model used in this article is what social scientists call the “natural science model” of social science . . . (Lee, 1989: 33–34)

In his article, Lee thus explicitly states that the natural science model of research that he describes is just one idealized model among many. He is describing an idealized model that is used in the social sciences, not the actual practice of natural science. Once again, we see that the authors are criticizing a straw man.

In conclusion, I agree with the first part of the authors’ argument. I agree that the natural sciences use qualitative as well as quantitative data, that the data they collect are subject to interpretation, and that they rarely find deterministic laws. This part of the argument is unproblematic. In fact I doubt if any IS scholars (at least the ones that that I know) would disagree with this. However, I disagree with the second part of their argument. They claim to be demystifying beliefs about the natural sciences in IS, but I believe they themselves have created a straw man. They claim to be correcting the erroneous views that IS scholars have held for years about the nature of the natural sciences, but they themselves have misunderstood what IS scholars actually said. Hence, they are not “demystifying beliefs about the natural sciences in IS” at all, but doing some mystifying of their own.