Abstract
We suggest that the organizational science’s increasing preoccupation with “interesting” theories and “counterintuitive” facts can lead to nonreplicable findings, fragmented theory, and irrelevance. The focus on the interesting and novel reveals a profound misunderstanding of the scientific enterprise. Organizational scholarship will be better off if it reverts to according primacy to the problem being solved over novel theory development.
In an essay published in the Organizational Psychology Review, Ferris, Hochwarter, and Buckley (2012) make the observation that our discipline’s emphasis on “revelatory contributions” to theory has led to a fragmented and disorganized body of research. They argue that the organizational science is full of theoretical models that do not build on one another. We agree. The state of our field and its obsession with novel theoretical contributions is antithetical to the goals of the scientific method (cf. Russell, 1931). As an ideal, the scientific method consists of a set of propositions arranged in a hierarchy, with the lower level propositions being logical deductions from higher level ones, and the higher level propositions being logical inductions from the lower level ones. The hierarchy also requires that the propositions are commensurable. As Ferris et al. (2012) eloquently summarize, the organizational sciences are far from this ideal and the excessive desire for theoretical novelty is largely to blame.
We offer a complementary view in this essay by suggesting that the field’s focus on the “interesting” (Davis, 1971) both in reporting on facts and in theories has also contributed to the fragmented state of the field. The demand that both theory and results should satisfy criteria of what is interesting, we argue, is antithetical to the scientific method and needs revisiting if we want to make progress. In particular, we contend that the focus on the “interesting” has mutated into a focus on finding counterintuitive “facts” and theories. This, in turn, creates two incentives to the scholarly community that are undesired from the standpoint of a positivistic model of science (Popper, 1935): (a) an incentive to backward-engineer theories to explain counterintuitive facts. The problem with this approach is that the resulting theories will have little explanatory reach and (b) an incentive to find facts that fit a counterintuitive theory. The problem with this approach is that the resulting facts will have little chance of replication.
We argue that the focus on the interesting is actually a misunderstanding of the scientific process. We also suggest that the adoption of Davis’s article as some kind of “how-to” conduct research is restricted to the organizational sciences. We believe that theories do not need to be interesting, new, or full of counterintuitive assumptions. Research problems, however, should be interesting and their definition should precede theory construction. We propose heuristics for how to define research problems.
The quest to be interesting
Davis’s (1971) article, “That’s Interesting: Towards a Phenomenology of Sociology and a Sociology of Phenomenology,” has been hailed as a classic and served as a basis for invigorating much organizational sciences enquiry. Arguing against what he considered to be mediocre and uninspired research in the social sciences, he exhorted scholars to understand what makes for interesting theories. Davis (1971, p. 328) claimed that,
Non-interesting social theories are often asserted on purpose by those who think that the business of social scientists is merely to assert any theory that can be derived and confirmed according to the text-book rules of theory construction and verification.
He went on to claim, (Davis, 1971, p. 328), that “the mediocre in the social sciences and (probably the natural sciences too) can be defined as those who take the textbook rules of scientific procedures too literally and too exclusively.” Davis was right in noting that a social scientist could have impact if they could take on and challenge the assumptions of their audience. But the seemingly casual dismissal of the scientific method as the province of the mediocre should provide a cautionary note against the fulsome acceptance of his ideas.
Rather than being cautious consumers, we think Davis's exhortations have been taken on a little too enthusiastically by organizational scientists in general and editors and reviewers of organizational journals in particular. It is noteworthy that the organizational sciences seem to be particularly swayed by his prescriptions. Specifically, an analysis of citation patterns of his article (Table 1) reveals that 71% of all citations the article picked up (260 cites in total) can be attributed to the organizational sciences. A hefty 83% of editorials citing the article have been published in organizational journals. This analysis suggests that the Davis article appears to have had a disproportionate impact on the organizational sciences. And the impact appears to be a function of the weight placed on the criteria espoused in the article by the editors of our journals. While some of these editorials (i.e., Eden & Rynes, 2003; Gill & Bhattacherjee, 2009; Markoczy & Deeds, 2009; Pearce, 2007; Schwarz & Huber, 2008) use the article to make the point about the kind of theories that make an impact, the others seem to understand it as a guide to construct theory or develop research problems. These editorials (e.g., Ahlstrom, 2011; Colquitt & George, 2011; Daft & Lewin, 1990; Reay & Whetten, 2011; Short, 2009) seem to communicate that articles need to be interesting in the Davis sense to warrant publication.
Pattern of citations for the Davis article in the Social Science Citation Index (SSCI) as on the 22nd of January 2013.
We agree that research problems must be interesting and attract the curiosity of the reader. But the interestingness comes about from describing a problematic state that demands explanation—for example, why is it that some studies find that network centrality is positively correlated with leadership success and others find that it is negatively related to leadership success? Mutually contradicting facts are an example of interesting problems that demand explanation as are facts that contradict theories. Interesting theories are not, by themselves, worthy of investigation unless they are put to the service of explaining research problems.
We think that Davis’s (1971) recognition of the distinction between interesting and truth— he says (p. 343), “I believe it is as important to learn why a theory is found interesting in some audience as it is to ascertain the truth of its content or the logic of its form.”—suggests the need to put his exhortation to be interesting in perspective. We are scientists first and salesmen second. If the order gets inverted, as we argue has become the case when “interesting” becomes the goal, we run the risk of perverting the scientific enterprise.
Our goal is not to argue for uninteresting work. We want to examine some of the consequences of the prevalent trend of elevating the interesting over the truth (and robust) and to sketch a route leading to a more balanced organizational science that is both interesting and robust. The time for reform is ripe because the organizational sciences have an opportunity to influence policy and practice in ways that we could not do in the past (Hambrick, 1994). Books based on scholarship in the organizational and other social sciences are becoming bestsellers; a perusal of the New York Times best-seller list for business books has Kahneman’s (2011) Thinking Fast and Slow at Number 3 and Chip and Dan Heath’s (2010) book Switch at Number 13. However, this influence is predicated on a robust body of knowledge and our view is that the focus on the interesting is preventing us from developing a replicable knowledge that can inform practice.
The persistent emphasis on the interesting is troublesome because research designed to uncover counterintuitive results is sure to find some (Nickerson, 1998). Without efforts to also examine the replicability of these results and their connections to theory, a distorted view of reality emerges that yields an absurd view of the world rather than usable guidance from thought and practice. In fact, if the social world was primarily counterintuitive, it would be an uninhabitable place in which people would have difficulty predicting what comes next. Surely, this is not the case—people are not in a permanent state of being puzzled by their counterintuitive surroundings.
The scientific method
Bertrand Russell’s (1931) summary of the scientific ideal is familiar to most readers of this journal. 1 But it bears repeating. Science, he says, starts from an observation of the particular, but is concerned essentially with the general. Empirical observation, no matter how interesting, is significant only to the extent that it helps to establish or refute some general law. Thus, the significance of any observed fact is relative to knowledge, and the validity of any law depends on the strength of the empirical evidence.
According to Russell (1931), the ultimate ideal of science is to have a set of propositions arranged in a hierarchy, with the lowest level of the hierarchy being concerned with particular facts, and the highest with some general law. The hierarchy is connected in upwards by induction and downwards by deduction. Specifically, if facts, A, B, C, D, etc suggest, via induction, a certain general law, then they are all seen as specific instances of the general law if the law was true. Similarly, another set of facts suggests, via induction, another general law, and so on. All these general laws suggest, by induction, a law of a higher order of generality. Now these laws are all instances of the more general law. There will be many such stages in passing from the particular facts observed to the most general law as yet ascertained. From this general law we proceed in turn deductively, until we arrive at the particular facts from which our previous induction had started.
How many of the empirical observations and theoretical propositions in the organizational sciences attempt to build towards this ideal? Not many. And a lot fewer with the current focus on novelty and interestingness. As Ferris et al., (2012) noted, the quest for revelatory theoretical contributions has led to fragmentation. Theories that are fundamentally different from one another cannot logically exist in the same hierarchy. How would you judge the significance of a theory if it is not being used (along with other theories) to formulate a more general theory? How would you also judge the validity of a theory if journals and editors ask for new theory in every paper rather than multiple tests of a theory?
Similarly, the quest for novelty and interestingness of facts has infused them with significance without any regard to the knowledge that they generate. Counterintuitive empirical findings are valued because they are counterintuitive rather than because they advance or validate a theoretical perspective. Also, they cannot be verified because our journals do not publish independent replications. And finally, their significance cannot be established as the significance of the theoretical perspective they advance can also not be verified as we already noted.
We would like to note that not all counterintuitive facts are without basis or usefulness in the social sciences. Quite the contrary! Some of the best-known theories in the organizational sciences and psychology have resulted from examining counterintuitive facts. We examine two of them in the next section and then explain how current views towards counterintuitive facts differ from the earlier research and why this approach is leading us to a dead end.
The role of counterintuitive facts
Research in the organizational sciences has resulted in a few stylized facts that inform further research and practices. The idea that individuals escalate commitment to a failing course of action is one of the better known such facts. Barry Staw’s (1976) classic paper on this idea provides an outstanding example of how an examination of the counterintuitive resulted in the formulation of a general model of investment behavior. Based on his observation that public failures in a policy (e.g., the U.S. involvement in Vietnam) appear to create pressure on the decision maker to justify their original decision, Staw hypothesized and designed a study to investigate the proposition that failures create a larger impetus for further investment than success. The results and the paper have had a lasting impact on the way we view investment decisions by managers and by organizations. They have also spawned an entire field of enquiry as evidenced by the many academic studies across different domains that followed from and built on this paper. What accounts for this success? First, the phenomenon was an interesting one and worthy of understanding in its own right. Understanding when people are likely to invest (time, money, and effort) in situations—when people increase investment in a decision despite new evidence suggesting that the current cost of continuing the decision outweighs the expected benefit—is of great practical importance. Second, many empirical studies that followed the Staw study replicated the main findings and extended them; journals and editors clearly provided more leeway for replication than seems to be the case now. And third, the theoretical explanation for this phenomenon was a special case of the well-established cognitive dissonance model.
This brings us to cognitive dissonance theory which is responsible for explaining some of the most interesting findings in social psychology. According to most accounts, the theory has its origins in Festinger’s attempts to explain the slightly bizarre set of circumstances surrounding a doomsday cult members’ deepening (rather than diminishing) faith following the failure of a cult’s prophecy that a UFO landing was imminent. In their book, When Prophecy Fails, Festinger, Riecken, and Schachter (1956) document how the believers met at a predetermined place and time, believing they alone would survive the earth’s destruction. When the appointed time came and passed without incident, rather than losing faith in the prophecy, most chose to believe that the aliens had given earth a second chance. Group members now thought that the group was empowered to spread the word that earth-spoiling must stop. Festinger and his colleagues concluded that when faced with acute cognitive dissonance (Had they been gullible simpletons? Had they given up their worldly ties and possessions in vain?), most individuals simply changed the meaning of the current set of circumstances and viewed the world failing to end as a sign to reaffirm their faith.
Cognitive dissonance theory, which explains this and other similar counterintuitive facts, has been a very successful one in social psychology. Why is this theory so successful? First, despite being based on counterintuitive findings, the theory itself is very intuitive as it posits that individuals seek consonance between their expectations and reality and that when dissonance occurs, they do one of three things, (a) lower the importance of one of the discordant factors, (b) add consonant elements, or (c) change one of the dissonant factors. Second, the theory explains several otherwise puzzling and seemingly irrational behaviors. And, finally, the theory could explain a broad range of behaviors ranging from reactions to unfair treatment at work (cf. Adams, 1965) to escalation of commitment to failing courses of action.
The cognitive dissonance example suggests that the examination of counterintuitive cases highlights factors that exist beyond the normal range. They therefore help magnify and make clear the facts that account for particular outcomes. Pondy (1979) provides a similar argument for the study of extreme cases.
The two examples provided before point to two different ways in which an examination of the counterintuitive can lead to robust scientific knowledge. In the case of cognitive dissonance, counterintuitive observations resulted in testing a robust theory that could explain many other important facts. In the case of escalation of commitment, an examination of the counterintuitive provided a robust understanding of the phenomenon which was interesting in its own right. However, in both cases, robust knowledge resulted from replication and refinement—an option that is increasingly not available in journals focusing on novelty and interestingness.
Consequences of the focus on the interesting
As we have argued, one of the consequences of our current focus on the counterintuitive along with a reluctance to publish replications is the increased fragmentation of our discipline. Another consequence is unreliable data. Researchers could fake and fabricate data as there is a bias towards novelty and there is no check that comes with a need to replicate. It is not just faked data that is a problem. A discipline that does not encourage replication and instead values the novel and the interesting invites ambitious academics to publish interesting one-off findings without any concern about scrutiny (cf. Rosenthal, 1979, on the bias towards publishing significant results).
A final consequence is that we are at risk of developing an absurd “science” of the counterintuitive. Most of the behavior in the world is intuitive and is still deserving of explanation and testing. By focusing on the counterintuitive, we fail to pay attention to a majority of behavior. If the counterintuitive behavior was important (as in the case of escalation of commitment) or used to test the robustness of a theory (as with cognitive dissonance), then we are all for examining it. However, the current obsession with searching for counterintuitive findings does not appear to serve either of these ends. It results in an organizational science that is fragmented. Worse, it makes it irrelevant.
As we have argued throughout this essay, the current trend towards “interesting” theories and counterintuitive facts does not allow for progress or for relevance. So what does? We think that Popper’s (1935) view that problem-solving is at the heart of the scientific enterprise provides the best way forward.
Towards a problem-centred organizational science
For a relevant organizational science that is also scientifically rigorous, the problem should come first, then the theory. The theory itself may be dull or old, but as long as the problem is one that is unsolved and of interest, it is demanding theory that explains it. The problem itself may come in one of three forms: 2
Two observed facts contradict each other (e.g., people like high performers but people also dislike high performers). A theory which explains both facts, even if very old, is a valuable one and a paper that examines it is a contribution.
A well-established theory is contradicted by a fact (e.g., the theory that people dislike high performers because they threaten their self-esteem is contradicted by the fact that people with high self-esteem also dislike high performers). The fact identified here is a significant one in Russell’s (1931) terms because it contributes to knowledge (in this case, providing some evidence that the theory is bounded or even wrong).
Two theories that explain the same fact contradict each other in their assumptions. (Theory A and Theory B both explain why people dislike high performers but Theory A assumes it is because high performance threatens people’s self-esteem and Theory B assumes it is because people are envious.) This is a colloquial restatement of the proposition that the examination of alternative explanations with a view towards provisionally accepting the validity of one over the other is good science.
The kinds of problems we describe before are clearly worth studying. They are likely to be important and generative as was the case with the escalation research that we described in an earlier section. Research that puts the problem first is scientific. Such research is likely to be interesting and scholars who examine problems are likely to have an impact.