Introduction: Replication of Crises - Interdisciplinary Reflections on the Phenomenon of the Replication Crisis in Psychology

The last decade has yielded some worrying diagnoses about the state of psychology. Psychology might be broken, some have warned (Woolston, 2015); It might even be trapped in an unpleasant space somewhere between the third and fourth circle of hell (Neuroskeptic, 2012). This state of affairs is probably known to almost everyone in academic psychology by now – the ‘replication crisis’ (variously also ‘replicability crisis’ or ‘reproducibility crisis’). Despite some conceptual differences, all these terms refer to the assumption that the results of many scientific studies cannot be reproduced, which has been supported by growing empirical evidence. Having already affected a range of disciplines in the medical and life sciences since the early 2000s, the replication crisis reached psychology around 2010. In many narratives, at least two widely discussed papers are often mentioned as major catalysts. In 2011, an article on ‘‘false positive psychology’’ was published, demonstrating the simplicity with which researchers could gather statistically significant evidence for a false hypothesis (Simmons et al., 2011). In the same year, an article in a prominent psychology journal, using recognized and widely established experimental-statistical methods, claimed to find evidence that humans are able to perceive the future (Bem, 2011). Since this time, the crisis has got more attention and was measured more precisely in its extent (Dominus, 2017; Morawski, 2019). In an attempt to replicate 100 experimental and correlational studies published in three psychology journals, only 36% of the results proved replicable (Open Science Collaboration, 2015). Many other major psychological findings could not be replicated, especially in social and partly in clinical psychology – from the widely published effects of power posing (Simmons & Simonsohn, 2017) to well-known findings on behavioural priming (Chivers, 2019); from the famous Marshmallow test (Watts et al., 2018) to the findings of clinical trials of pharmacotherapy relative to psychotherapy (Hengartner, 2018).

Diagnoses of a replication crisis in psychology were accompanied early on by suggestions of potential remedies (e.g. Munafò et al., 2017; Nosek & Lakens, 2014). Most of these remedies were aimed at necessary technical reforms of scientific practices in psychology, and the scientific system as a whole – problems in quantitative-experimental methods, data analysis, publication practices and incentive structures. The reform proposals included, for instance, more methodological rigour and preregistration in journals to prevent p-hacking, the use of a replication-index and the adoption of open science practices. But many of the attempts at reform focused primarily on scientific methods and on techniques of quality control, while sometimes also addressing some contextual issues such as incentive structures (Begley et al., 2015) and research culture (Munafò et al., 2020).

However, relatively little attention was paid to larger contextual factors of the crisis – to epistemic cultures, philosophical analyses, historical reconstructions and theoretical considerations. Moreover, amidst the initial debates about methodological flaws and possible reforms, a reflexive perspective on the debates themselves was still lacking. Only in the last 3 years have some important contributions from the humanities and the history and philosophy of science on the replication crisis in psychology appeared, which have added a vital reflexive layer to the debate (e.g. Feest, 2019; Flis, 2018, 2019; Morawski, 2019; Wiggins & Chrisopherson, 2019).

In this Special Issue, we believe that such a reflexive discussion that includes contextual aspects is crucial and deserves to be continued and intensified. With the plural ‘replication of crises’ in the title, we want to make clear that the current crisis is more than one. What is perceived as a crisis – and whether it is perceived a crisis at all – varies depending on the scientific field, theoretical background, and epistemological assumptions. Already, the history of psychology was closely connected with the proclamation of various crises, and the current replication crisis is not only a methodological one but also a narrative, cultural, epistemological and political event that needs a closer examination. As a consequence, we want to promote both an intra-disciplinary dialogue between scientific and theoretical psychology, and an inter-disciplinary dialogue between psychology, the humanities and the social sciences.

Our Special Issue grew out of a workshop held at the University of Lübeck in Germany in 2019, which aimed to unite scholars from both scientific psychology and from fields that are more influenced by the humanities and science studies. The workshop was organized by Lisa Malich, Eva Barlösius, Cornelius Borck, Uljana Feest, Jonas Obleser, and Torsten Wilholt – presenters included Marcus Munafò and many of the authors of this Special Issue such as Simine Vazire, Nicole Nelson, Maarten Derksen, or Jill Morawski. In the various contributions on the following pages, we want to foster multiple reflective perspectives on the phenomenon of the replication crisis, following three main questions:

1. What is specific about the replication crisis in psychology?

Several articles focus on the replication crisis within the epistemic culture of psychology, addressing concrete functions, objects and practices of replication, as well as key self-narratives and debates of what psychological research actually is. In her historical study, Annette Mülberger investigates how replication was employed as a scientific tool by early experimental psychologists in the late 19^th and early 20^th century. Exploring the research practices by influential psychologists like Fechner, Ebbinghaus, Muller, Wundt, or Titchener, she shows that replications were only sometimes about simple repetition and confirmation of previous experimental results; rather, the purposes and forms of replication were more diverse, with some experiments aiming from the outset at different results than the original study. The article by Ivan Flis examines historical discourses from scientific psychology in the 20th century and detects that they contain certain unquestioned ideas about the autonomous role of scientific literature, which Flis calls the ‘myth of self-organization’. Flis discusses the continuing impact of this myth in contemporary debates about the replication crisis and draws parallels to preregistration and theory formalization suggested by psychology’s reform movement.

Current psychological debates of the replication crisis are the focus of the following two articles. Jill Morawski analyses how psychologs objects are conceived to reconstruct different ontological assumptions. In so doing, Morawski distinguishes two discourses: that of the reformers, who assume stable and singular objects, and that of the challengers, who presume complex and contextual objects. Furthermore, she demonstrates that ontological claims are linked to specific methodological arguments and notions of the scientific self. Maarten Derksen and Sarahanne Field also discuss constructions of the scientific self in psychology in their article, but take a different starting point: they examine the so-called tone debate, which is about how psychologists should express criticism in the context of the replication crisis. Derksen and Field trace not only how the relationships between knowledge, social order and subjectivity are redefined in these debates but also what dynamics are set in motion by new social media technologies such as Twitter.

2. What are the connections between the replication crisis in psychology and that in other scientific fields? Are there underlying common mechanisms in science?

Several papers measure the tension between the plurality and the generalizability of the replication crisis. Depending on the field of knowledge, the crisis can have specific trajectories, take different forms, and thus represent different crises. At the same time, however, some crises can be characterized by common dynamics and allow general statements about contemporary science. In their article, Nicole Nelson, Julie Chung, Kelsey Ichikawa, and Momin Malik focus on the similarities between psychology and other fields that were affected by the replication crisis almost simultaneously. Using a mixed methods analysis of commentaries on replication in psychology and the biomedical sciences, they point out that these narratives share specific arguments. The authors attribute these communalities to common historical, cultural, or institutional factors driving discontent with established scientific practices. Felicity Callard then takes an in-depth look at one of these possible common factors: academic labour. Arguing that the replication crisis can be understood as a crisis of social reproduction at universities, she highlights the interconnectedness between the financial crisis of 2007–2008 and the subsequent intensified precarization of various forms of academic labour. Such politics of occupational insecurity increase the pressure created by incentive structures in academia, which can, for example, raise the likelihood of scientific misconduct.

The next two articles take a more epistemological and methodological turn and deal with general scientific practices and models. Asking how to evaluate if a particular scientific field has effective self-correction mechanisms, Simine Vazire and Alex Holcombe develop Observable Self-Correction Indicators (OSCIs). These comprise two dimensions: first, transparency, which is already addressed in most reform movements and second, critical appraisal, which is still often neglected and which the authors define as automated error detection, computational and empirical reproducibility checks, pre- and post-publication peer review, a critical approach to literature, strong theory, and diversity. Brian Haig takes a different angle in his article, questioning whether replication is even as central to science as it is often portrayed in the context of the replication crisis. In doing so, he critiques some common assumptions about replication and argues for paying more attention to triangulation and adopting a more complex approach to theory construction in science.

3. What are possible underlying causes of the replication crisis in psychology, and what are the opportunities for improvement?

After the examination of more general scientific principles, the last three articles refocus psychology. The search for reasons and resulting reforms has been integral to discussions of the replication crisis in psychology from the beginning. In this regard, some essays in the Special Issue break new ground and add their own perspectives to existing debates. Mario Gollwitzer and Johannes Schwabe advocate paying more attention to context dependency in psychological replication research. To this aim, they develop an analytical framework, in which contextual characteristics are conceptually relevant when the observed heterogeneity of effect sizes cannot be attributed to psychometric properties. To illustrate their framework, Gollwitzer and Schwabe refer to and reinterpret the example of the ‘moral typecasting’ effect, which is part of the ManyLabs2 replication project. In contrast, Wolfgang Maiers turns to more general aspects and challenges current psychological research as a whole. Drawing partly on the theoretical concepts of German Critical Psychology, he argues that experimental research structurally excludes the subjective, although it is a central dimension of the human psyche. Against this background, Maiers views the current replication crisis as a continuation of the various earlier crisis discussions in the history of psychology, behind which there lies actually the failure of a ‘psychology without subject’. In their article, Lisa Malich and Christoph Rehmann-Sutter discuss the umbrella approach of metascience as a response to the replication crisis. After emphasizing the epistemic gains of many metascientific endeavours, they address some critical points, in particular the reductionist view of science that underlies many metascientific concepts. They therefore argue for a second, supplementary umbrella approach, that of psychological humanities. In order to outline how the psychological humanities can provide a valuable response to the replication crisis, they address questions of validity as well as ethical, social, and epistemological issues.

Overall, this Special Issue encompasses heterogeneous perspectives that represent different epistemic points of view. Many will be challenging, and will provoke debate. The arguments in some articles might be in tension with each other – yet it is perhaps precisely this diversity that can turn what some have described as a crisis into an opportunity. If we can reflect and learn, we can improve.