Abstract
Why did Japan conclude the Japan–Soviet Neutrality Pact in 1941, even though it knew the German-Soviet relations were deteriorating and heading towards a disastrous war? Under the Tripartite Pact, it would be irrational for Japan to approach the USSR, which Germany had decided to invade. This article examines this long-standing puzzle in Japanese diplomatic history (also an anomaly of neorealism) by developing a new neoclassical realist model termed ‘neoclassical realist model of overconfidence’ based on self-deception in evolutionary psychology using scientific realism in the philosophy of science as a metatheoretical foundation. Drawing on neoclassical realism, I argue that Japan’s balancing strategy during 1940–41, which initially reflected the tripolar structure of the international system (independent variable), ultimately resulted in the suboptimal balancing of the Japan–Soviet Neutrality Pact (dependent variable) due to Yosuke Matsuoka’s (Japanese foreign minister) self-deception, including the positive illusion and cognitive dissonance effect (intervening variable).
Keywords
Introduction
Why did Japan conclude the Japan–Soviet Neutrality Pact in 1941, even though it knew the relations between Germany and the Union of Soviet Socialist Republics (USSR) were deteriorating and heading towards a disastrous German-Soviet war? 1 In a situation where Japan has concluded a military alliance (i.e. the Tripartite Pact) with Germany, it would be irrational for Japan to approach the USSR, which Germany had decided to invade. If so, how can realist theories provide such an irrational behaviour viewed as a suboptimal balancing with scientific explanations?
This article examines the longstanding historical puzzle, which is also an anomaly of neorealism, by developing a new neoclassical realist model termed the ‘neoclassical realist model of overconfidence’ based on self-deception in evolutionary psychology 2 using scientific realism in the philosophy of science as metatheoretical foundations. 3 Based on neoclassical realism, I argue that Japan’s balancing strategy during 1940–41, which initially reflected the tripolar structure of the international system (independent variable), ultimately resulted in the suboptimal balancing of the Japan–Soviet Neutrality Pact (dependent variable) due to a sequence of Yōsuke Matsuoka’s (Japanese foreign minister) self-deception, including the positive illusion and cognitive dissonance effect (intervening variable). 4 This paper aims to clarify the causal mechanisms through which Japanese leaders irrationally adhere to their approach towards the USSR even though the initial strategic rationale for the rapprochement with the USSR (i.e. the realisation of the four-power entente, including Germany, Italy, Japan and the USSR) had been lost due to the deterioration of the German-Soviet relations leading to the catastrophic war.
Theoretically, overconfidence has long been regarded as a significant cause of foreign policy failures (including suboptimal balancing) by well-known scholars (e.g. Geoffrey Blainey and Stephen Evera). 5 Nonetheless, realist theories have been unable to theorise overconfidence with adequate scientific foundations because they (1) ignore one of the crucial psychological findings on overconfidence, namely self-deception (i.e. the psychological mechanism of denying the importance or significance of opposing evidence and logical argument) and (2) fail to incorporate overconfidence into the realist research programme in a way that consciously addresses the level of analysis problem (i.e. lack of neoclassical realist models explaining overconfidence). This article aims to fill the gaps by constructing the neoclassical model of overconfidence, which explains suboptimal balancing behaviours driven by leaders’ overconfidence, by introducing self-deception in evolutionary psychology into the realist research programme. As a plausibility probe to show that the argument is sufficiently grounded in the evidence to justify further research, 6 this paper illustrates the new model by examining the case of the Japan–Soviet Neutrality Pact in 1941 with important historical sources and literature in Japanese, offering novel psychological (also neoclassical realist) insights into the crucial case during World War II. 7
The article has five sections. The first section explains evolutionary psychological findings on self-deception as the intervening variable of the new model. The second section develops the neoclassical realist model of overconfidence by introducing self-deception into the realist research programme. The third section provides the case study of the Japan–Soviet Neutrality Pact to illustrate the new model as a plausibility probe. The fourth section responds to criticisms against the new model based on scientific realism. The concluding section summarises the article and explains the findings, implications and limitations of the research for international relations (IR) and practice.
Why do policymakers often fall prey to overconfidence?
Overconfidence is the product of self-deception, the psychological mechanism of denying the relevance or significance of opposing evidence and valid arguments. 8 Self-deception causes humans to overestimate positive events and the possibility of control over situations and underestimate their susceptibility to various hazards. This unreasonably optimistic view of oneself, which is the effect of self-deception, is called the positive illusion. 9 Humans, for example, tend to underestimate the likelihood of contracting cancer and being involved in traffic accidents but exaggerate the likelihood of their longevity and professional success. They also tend to overestimate the groups to which they belong, perceiving in-group individuals to be superior to the population average, even if they are random strangers. 10 Psychologist Goleman states that ‘[a] well-functioning group is bound together by a kind of group narcissism, one that subscribes to the familiar positive illusions: an unrealistically positive sense of itself, the somewhat grandiose sense of how much the group can make a difference . . . and an overly optimistic sense that things will turn out well’. 11 Therefore, overconfidence, regardless of whether it stems from individuals or groups, is the product of self-deception.
If so, what does self-deception stem from? The answer lies in evolution.
12
Self-deception is a product of evolution by natural selection. Human brains are equipped with self-deception because it was favourable for survival and reproduction in the environment of evolutionary adaptedness (EEA), where various psychological mechanisms (i.e. human nature) evolved during the hunting and gathering age.
13
One of the adaptive roles of self-deception is effective persuasion in social interaction: if you can convince yourself that you are speaking the truth, you may persuade others more effectively. In other words, if you want to convince people efficiently, you need really believe what you are saying and be confident in your own. Trivers, a pioneer in the self-deception theory and a renowned evolutionary psychologist, states its essence as follows:
We deny the truth to ourselves. . .We repress painful memories, create completely false ones, rationalise immoral behavior, act repeatedly to boost positive self-opinion, and show suite of ego-defense mechanisms.
14
Self-deception is comparable to the visual illusion of being the result of evolution by natural selection. Due to natural selection, our brains have evolved to view the world in a certain manner; thus, we cannot escape this illusion easily. Similar reasoning of the logic of evolution lies in self-deception. 15 For instance, many individuals unconsciously tend to exaggerate their attractiveness. Behavioural scientists Epley and Whitchurch provided respondents with three sorts of images of their faces: real, attractive and ugly. The findings demonstrated that they reacted to attractive, real and ugly faces from the quickest to the slowest. 16
According to rational decisionmaking theories (e.g. rational choice theories and bayesian theories), people update their beliefs when their expectations conflict with reality. 17 However, research on self-deception suggests that people are not as reasonable as previously believed. Although they seldom learn bad (in a subjective sense) information, they learn good (in a subjective sense) information and conveniently reinterpret inconvenient facts to fit into existing beliefs. This tendency to selectively update one’s belief in a self-serving manner is called the cognitive dissonance effect, another consequence of self-deception. 18 Neuroscientist Sharot, for instance, asked participants to estimate the likelihood of contracting the disease, informed them of the actual probability and studied how they adjusted their original estimates. Consequently, respondents who estimated the likelihood to be 50% and got accurate information that the probability was 30% (subjectively good information) adjusted their original prediction to 35% (appropriate updating). In contrast, participants who first estimated the likelihood to be 10% but got accurate information that the probability was 30% (subjectively bad information) subsequently revised their estimate to 11% (irrational persistence with the original estimate). 19
Although self-deception is present in all individuals more or less, narcissistic personality actors strongly exhibit this attribute. 20 Approximately 1% of the population is equipped with this acute self-deception. 21 Although self-deception sometimes leads to the ‘evolutionary mismatch’ 22 in the modern world due to the difference between the EEA and the contemporary world, the adaptive advantage of self-deception is still (to a certain extent) present today because it has precisely contributed to the survival and reproduction of our ancestors during the EEA. 23 As the remaining 99% seldom meet these deviant actors with powerful self-deception in evolutionary history, natural selection has not favoured psychological mechanisms that can resist narcissistic personality actors with strong self-deception. 24
Neoclassical realist model of overconfidence
The preceding section explored self-deception in evolutionary psychology. However, when developing realist models theorising psychological factors, IR theorists need to ensure that self-deception itself is not an IR theory but a micro factor of psychology. 25 In other words, there is a risk of falling into what philosopher Dennet referred to as ‘greedy reductionism’ (i.e. the attempt to explain everything in terms of its simplest constituents) when theorising social phenomena based on micro factors of natural science while ignoring macro factors of international relations, such as realist assumptions of anarchy. 26
Neoclassical realism, which takes the level of analysis problem seriously, has the advantage of overcoming this trap of greedy reductionism. According to neoclassical realism, balancing strategies that leaders adopt are not direct products of systemic factors but outcomes of the mediating process by which ‘the state perceives them and responds to them within the institutional constraints of its unique domestic circumstance’. 27 Neoclassical realism differs from other realist schools primarily in the degree to which the translation of systemic factors into state behaviours is explicitly analysed rather than assumed a priori or treated as distinct spheres of inquiry. According to neoclassical realism, adequate accounts of balancing behaviour require incorporating unit-level intervening factors influencing leaders’ choice of balancing strategies. Neoclassical realism can explain variations of foreign policy (i.e. dependent variables) across time and space by supplementing structural factors (i.e. independent variables) with domestic factors (i.e. intervening variables) mediating the causal effects of the international system. If so, how can neoclassical realists utilise domestic factors in a methodologically conscious manner?
To deal with this problem, this article constructs the new neoclassical realist model by introducing self-deception into the realist research programme based on the methodology of scientific research programmes (MSRP) in the philosophy of science. 28 By doing so, scholars can form a bridge between micro factors of psychological findings and macro contexts of international politics. According to the MSRP, the following are the steps for developing the new model. The first procedure is to assume the validity of the realist hardcore a priori. 29 There are four cores in the realist research programme. 30 First, human beings do not primarily face each other as individuals but as members of groups pledging fidelity towards states. Second, international affairs take place under anarchy. Third, the nature of international politics is essentially conflictual. Fourth, power is a fundamental feature of international politics. The second procedure is to add the novel variable (i.e. self-deception) as an auxiliary assumption to explain the anomaly of realism (i.e. suboptimal balancing due to overconfidence). When doing so, this study does not aim to reject/refute the hardcore of the realist research programme but attempts to expand its explanatory power by adding the novel auxiliary assumption while accepting the minimal loss of theoretical parsimony.
The neoclassical realist model of overconfidence holds the standard neorealist assumption: the primacy of anarchy in international politics. 31 In other words, in most cases, states behave according to the dictates of the system pressure. What is important for neoclassical realists, therefore, is how to choose domestic factors (intervening variables) mediating between the structural constraints (independent variable) and balancing behaviours (dependent variable). 32 The neoclassical realist model of overconfidence consists of two hypotheses derived from the intervening variable of self-deception. The first is the positive illusion hypothesis. When assessing external situations, leaders sometimes overestimate the power of their states/alliances (i.e. relative and bargaining power) and the possibility of control over situations (e.g. averting wars) while underestimating the opposites (the power of enemies, risk of losing reputation from alliances and inconvenient information including poor military and financial situations). This cognitive asymmetry between favourable and unfavourable information is an anomaly of rationalist theories, and it is the strength of the neoclassical realist model of overconfidence that can explain it.
The second is the cognitive dissonance hypothesis. Faced with undesirable events (including policy failures and poor external situations), leaders sometimes remain overconfident by conveniently reinterpreting adverse events/policies to fit into existing beliefs. The cognitive dissonance effect (i.e. reinterpreting the value of suboptimal options to resolve the psychological conflict stemming from the choice) is particularly salient when causing adverse events or selecting options by choice. If the information is constant, rationalist theories cannot explain the preference reversal (caused by own choice) because they usually assume that ‘preferences affect actions’, not vice versa. 33 In contrast, the neoclassical realist model of overconfidence can explain this anomaly in terms of the cognitive dissonance effect.
The neoclassical realist model of overconfidence contributes to neoclassical realism by adding a new model to its repertoires from the novel perspective of overconfidence. In the past, neoclassical realists have theorised domestic factors from various perspectives (including strategic culture, 34 risk, 35 and domestic politics 36 ). However, previous research has failed to develop a neoclassical realist model theorising overconfidence, as it misses the scientific basis for overconfidence, namely self-deception. The significance of the neoclassical realist model of overconfidence lies in its filling the gap. Among neoclassical realist models, it is what Taliaferro et al. dub the ‘Type 1’ (neoclassical realism as a guide to explaining anomalies), 37 which focuses on leaders’ factors as intervening variables. Since the most biologically plausible explanation for the irrationality of leaders’ behaviours is what focuses on psychological factors (i.e. misperception), the neoclassical realist model of overconfidence, which introduces self-deception, is one of the promising realisations along with Taliaferro’s balance of risk theory. 38
Before proceeding to the case study, three caveats are worth noting. First, as neoclassical realism, the neoclassical realist model of overconfidence holds that the parameters of states’ balancing behaviours are dictated by the incentives and constraints of the international system, including the distribution of relative power, offence-defence balance, alliance structure, geography and the balance of power. 39 The effects of self-deception (i.e. positive illusion and cognitive dissonance effect) as intervening variables mediating systemic factors are causally significant only when balancing behaviours deviate from the dictates of the international system. After all, the ‘first cut’ in the analysis of the neoclassical realist model of overconfidence is structural factors, which is a significant difference from other theoretical approaches (e.g. middle-range theories not labelled as realism, foreign policy analysis and liberalism) as explored in the Folker’s analysis of the primacy of environment in realism. 40
Second, two hypotheses of the neoclassical realist model of overconfidence (positive illusion and cognitive dissonance hypothesis) are essentially ideal types, which are not necessarily mutually exclusive. For example, leaders may overestimate their state’s power while conveniently reinterpreting inconvenient information about their weaknesses as evidence of strength. Nevertheless, as ideal types, there are differences between the two hypotheses. The positive illusion hypothesis emphasises the cognitive asymmetries in perceiving positive and negative information, while the cognitive dissonance hypothesis emphasises the convenient reinterpretation of information to fit into existing beliefs. Therefore, it is reasonable to regard them as independent as ideal types.
Third, this article does not aim to generalise or test the neoclassical realist model of overconfidence inductively by case studies. Instead, the following section illustrates this model by examining the case of the Japan–Soviet Neutrality Pact as a plausibility probe to show that the argument is sufficiently grounded in the evidence to justify further research. 41 Plausibility probes are qualitative case study methods, which are not strict case-testings but ‘preliminary studies on relatively untested theories and hypotheses to determine whether more intensive and laborious testing is warranted’. 42 Since the neoclassical realist model of overconfidence is theoretically novel and untested for now, a plausibility probe is appropriate at this research stage. The purpose here is to illustrate the logic of the new model to serve as a starting point for future research. The case is selected due to its intrinsic importance 43 and suitability for the plausibility probe. 44
The Japan–Soviet Neutrality Pact in 1941
Systemic factors as a ‘first cut’ of analysis
According to neoclassical realism, the general framework of state behaviour is determined by the constraints and incentives of the international system, which are then mediated by domestic intervening variables, resulting in the actual balancing strategy. It is true that the Soviet-Japanese Neutrality Pact in 1941 was the suboptimal balancing for Japan, concluded despite the high probability of the German-Soviet war. However, this is partially due to the hindsight of history, and at least until the probability of the German-Soviet war increased, the Neutrality Pact had some strategic rationale. In other words, the Pact may have somewhat reflected the structural incentives of the time. If so, what were the structural conditions of the international system, which serve as the ‘first cut’ of the neoclassical realist model of overconfidence, leading to the Soviet-Japanese Neutrality Pact? The following sections reconsider the international context and two rational explanations derived from structural factors. 45
The post-World War I world had become a unipolar system with the United States as the sole pole with its influence growing in place of the exhausted European powers. 46 The 1920s world remained relatively stable as long as the unipolar system of status quo states continued. In this international context, Japan adapted to the unipolar system. On the issue of Siberian mobilisation after the Russian Revolution, Japan had carefully sought the American position on the issue, both at the time of the early British and French requests for mobilisation and at the Allied proposals for mobilisation on 11 March and 7 June 1918, with the attitude that Japan’s mobilisation would depend on the wishes of the United States. The Japanese Government and Army also unanimously adopted a coalition approach, avoiding a unilateral deployment. 47 Under Foreign Minister Kijurō Shidehara, Japan implemented a series of status quo policies, including a policy of non-interference in political disputes on the Chinese mainland. Indeed, from 1927 onwards, China’s provocative attitude towards Japanese interests greatly irritated Japan (especially the local Kwantung Army, namely Japanese armed forces in Manchukuo), and the international system was transitioning from unipolar to bipolar (the United States and the USSR). 48 However, after the Geneva Conference in 1927 and the London Conference in 1930, the Japanese government still maintained a course of international cooperation despite domestic political difficulties and the shift in polarity in the international system.
From 1928 onwards, with the emergence of the USSR as a revisionist superpower, the international situation gradually changed. Between 1928 and 1935, as the USSR embarked on the most prominent military build-up in its history, its relative position in the international system rapidly improved. Meanwhile, the Great Depression occurred in 1929, and the Manchurian Incident broke out in 1931. In 1935, with the rise of Nazi Germany, according to neoclassical realist Schweller, the international system shifted into the tripolar structure (i.e. two revisionist powers, the USSR and Germany, and one status quo power, the United States), which was inherently unstable. 49
In March 1936, Germany defied the Locarno Treaty, entered the Rhineland, annexed Austria in March 1938, and threatened to annex the Czech region of Südeten. Eventually, Britain and France gave in to German threats (Munich appeasement), but Germany was not satisfied and occupied the entire Czech Republic in March 1939. Then, in August of the same year, the German-Soviet Non-Aggression Pact was concluded, and with the German advance into Poland on 1 September, the Second World War began. In this context, Japan chose to bandwagon with the revisionist states. It is true that the German-Japanese Anti-Comintern Agreement of 25 November 1936 and the German-Japanese-Italian Anti-Comintern Agreement of 6 November 1937 had little substantive meaning due to the reservation conditions attached by the Japanese side. However, from the outside world, the Anti-Comintern Agreements gave a strong impression that Japan had joined the Axis bloc. 50 Even in the phase following the outbreak of the Sino-Japanese War in July 1937, Japan’s policy towards Germany was quite cautious, partly because Germany continued to provide aid to China until May 1938. 51 Although Germany, planning a war in Europe, expected the Imperial Japanese Navy to play the role of a check on the Royal Navy, the Japanese government was wary of getting involved in a war in Europe. However, when World War II began in September 1939, and the Netherlands and France fell by the following summer of 1940, the slogan ‘Don’t miss the bus’ became popular in Japan. 52 The military also began to focus on the French and Dutch colonies in South East Asia that had surrendered to Germany and their strategic resources, and the Konoe Cabinet, formed in July 1940, confirmed within the Cabinet the strengthening of the alliance with Germany and decided to bandwagon to Germany in order to benefit from the Dutch and French colonies. On 27 September 1940, the Tripartite Pact between Germany, Italy and Japan was formed.
The four-power alliance or the four-power entente (Germany, Japan, Italy and the USSR) had originally been conceived by Ribbentrop and others in Germany. Matsuoka later negotiated with the USSR, believing that it would succeed. However, the German-Soviet negotiation had failed before the Stalin-Matsuoka talks because Hitler was already preparing for war with the USSR. In June 1941, Germany started the war with the USSR, which implied that, under the tripolar system, Germany could turn two poles against each other. Japan, nonetheless, failed to realise the implications of the German-Soviet war and ended up starting a war with the United States in December of the same year. Although the United States adopted a back-passing strategy (especially on the European front) until as late as June 1944, 53 the second world war ended with the surrender of Germany and Japan in 1945 due to American intervention. Why did Matsuoka underestimate the possibility of the German-Soviet war and conclude the Soviet-Japanese Neutrality Pact, the suboptimal balancing strategy that led Japan to the Pacific War? It is one of the significant puzzles in Japanese diplomacy during the second world war, yet it remains theoretically and empirically unresolved.
When neoclassical realism discusses causality, the descriptive meaning of setting the causal effects of the international system as the independent variable of state behaviours is that the balancing strategies in question have certain rationality that reflects structural constraints and incentives. 54 What, then, can be attributed to the rationality of Japan’s conclusion of the Soviet-Japanese Neutrality Pact based on the aforementioned international context? Here, as a ‘first cut’ of the neoclassical realist model of overconfidence, this paper deals with two rational explanations derived from existing realist theories: offensive realism and the wedge theory.
According to offensive realism, all states are rational revisionists that make worst-case assumptions about enemies’ present and future intentions. In such a dangerous world (as cooperation is a risky option that can induce exploitations from potential enemies), rational behaviour is relative power maximisation for states. Additionally, offensive realism predicts (when encountering enemies) that states do not take conciliatory policies (e.g. arms reduction, appeasement and costly signalling) but hardline ones (e.g. war and internal balancing). 55
Interestingly, such an offensive realist logic can be seen in the four-power entente scheme (Germany, Italy, Japan and the USSR) directed against Britain and the United States, which formed the basis of Japanese diplomacy leading to the Soviet-Japanese Neutrality Pact. Hosoya, an expert on Matsuoka’s diplomacy and a leading scholar of Japanese diplomatic history, outlines this scheme as follows.
First, the Tripartite Pact was to be established. Next, Japan will use the power of this alliance to realise the four-power entente scheme between Japan, Germany, Italy, and the USSR. In doing so, Japan would use Germany’s ‘influence over the USSR’, and have Germany play the role of mediator in the adjustment of diplomatic relations between Japan and the USSR. Furthermore, once the four-power entente is established, Japan will use its coercive power to launch negotiations with the United States to conclude various issues and force it to withdraw from its interventionist policies in Asia and Europe and commit to joint efforts to restore peace in these regions. In the meantime, the Tripartite Pact and the four-power entente would be used to check Britain and the United States and promote Japan’s policy of southward expansion. This way, the spheres of influence in Europe, Asia, and Africa will be divided among the four powers (the United States, Germany, the USSR, and Japan), and new world order will be established.
56
As suggested by Hosoya’s analysis, offensive realism can explain why Matsuoka attempted to balance against the United States by utilising the Tripartite Pact and its logical consequence of the four-power entente (though it did not realise). For example, Matsuoka suggested a hardline policy, arguing that ‘it is an urgent business of ours to confront the United States’. 57 Under the tripolar structure of the international system, it is rational for states to balance against one pole while allying with the other two poles. In this sense, it is a kind of opportunistic bandwagoning, embodied by the Japanese slogan of the day, ‘don’t miss the bus’. 58 Again, Hosoya explains this point as follows.
[T]he rising prestige of Germany in the eyes of the Japanese resulted in resurrecting pro-Nazi sentiment from its demise following the conclusion of the Non-aggression Pact. This change in public opinion naturally affected the balance of power between the Anglo-American and Axis factions in Japan. Second, the existence of the French and Dutch colonies in Indochina and the East Indies now swam into the ken of the Japanese people, and a mood to seize the opportunity to advance into Southeast Asia spread to all strata of society.
59
Besides, Taliaferro explains that Japan’s conclusion of the Japan–Soviet Neutrality Pact is consistent with the strategic logic of offensive realism because ‘[b]y stabilising the Manchurian-Soviet border, the pact decreased the likelihood that a minor border dispute might erupt into a full-scale war’. 60 However, offensive realism cannot explain why Japan decided to conclude the Japan–Soviet Neutrality Pact despite salient information about the deterioration of the Germany-Soviet relationship because structural theories cannot explain leaders’ psychological biases (including self-deception), which cause suboptimal balancing behaviours.
The second rational explanation reflecting the situation of the international system is derived from the wedge theory. Wedge strategy is defined as ‘a state’s attempt to prevent, break up, or weaken a threatening or blocking alliance at an acceptable cost’. 61 Crawford, a proponent of the wedge theory, regarded Japan’s conclusion of the Japan–Soviet Neutrality Pact as an anomaly of neorealism, explaining this phenomenon in terms of the wedge theory rationally. Surely, a certain strategic rationale exists for Japan to wedge between China and the USSR, thereby isolating Chiang Kai-shek. 62 Nonetheless, as Hosoya points out, it was no longer timely because the United States and Britain made their attitudes towards helping China clear in response to the conclusion of the Tripartite Pact in 1940. 63 Thus, certain inherent limitations lie in the rationalist explanation of the wedge theory.
After all, the two rational explanations at the system level can explain the longstanding strategic motive (i.e. independent variable) behind the Japanese balancing strategy regarding the Japan–Soviet Neutrality Pact. Nonetheless, these system-level factors cannot explain why Matsuoka pursued a settlement with the USSR despite the high probability of the German-Soviet war and the resultant declining strategic rationale for the reconciliation with the USSR (because the four-power entente scheme had become impossible to realise). In other words, system-level explanations cannot solve the crucial puzzle of the interwar Japanese diplomatic history: why Japanese policymakers received information suggesting a high probability of the German-Soviet war, yet persistently ignored it. Rational theories cannot scientifically explain the psychological aspects of this case (i.e. the salient overconfidence of Matsuoka), which produced Japan’s suboptimal balancing (i.e. the conclusion of the Japan–Soviet Neutrality Pact), because of the ignorance of the evolutionary psychological findings on self-deception.
If system-level theories are universally applicable (i.e. states always behave according to the dictate of the international system), it would be reasonable for Japanese policymakers to admit that the four-power entente is impossible to achieve and stop pursuing a rapprochement with the USSR as soon as they perceive the high possibility of the German-Soviet war in 1941. In other words, the pursuit of reconciliation with the USSR and the resultant conclusion of the Soviet-Japanese Neutrality Pact under the certainty of the outbreak of the German-Soviet war cannot be considered a behaviour sorely reflecting structural imperatives, so some domestic variable needs to be working to mediate the causal effects of the international system. 64 At this point, Matsuoka’s self-deception is the domestic intervening variable that mediates the system pressure.
Matsuoka’s self-deception as a domestic intervening variable
Why did Japanese leaders conclude the Japan–Soviet Neutrality Pact even though they knew the German-Soviet relation was deteriorating? In a situation where Japan has concluded the Tripartite Pact, it would be irrational for Japan to approach the USSR, which Germany was eager to invade. Self-deception (a domestic intervening variable of the neoclassical realist model of overconfidence) can solve this historical puzzle from positive illusion and cognitive dissonance reduction perspectives. The overall logic is as follows. Driven by self-deception, Matsuoka overestimated the possibility of realising the four-power entente and underestimated the information about the deteriorating relationship between the USSR and Germany, sticking to rapprochement with the former. Eventually, he chose the suboptimal balancing policy of the Japan–Soviet Neutrality Pact.
According to the two hypotheses of the neoclassical realist model of overconfidence, the aforementioned logic can be theoretically divided into two psychological explanations. First, according to the positive illusion hypothesis, Matsuoka was overconfident about his bargaining power against the USSR, German pressure towards the USSR and the possibility of averting the war between the USSR and Germany, while underestimating the bargaining power of the USSR against Japan, the risk of the deterioration of the Japan-Germany and Japan-U.S. relationships caused by the rapprochement with the USSR, and the possibility of the German-Soviet war. Second, according to the cognitive dissonance hypothesis, although Matsuoka desired the four-power entente, a stronger alliance with the USSR than the neutrality pact, he was forced to conclude the latter. To cope with the psychologically stressful situation, he remained optimistic by positively reinterpreting the strategic importance of the neutrality pact. The following illustrates how these causal mechanisms worked in the actual historical situation by process tracing while examining Matsuoka’s particular role in the Japanese government and alternative intervening variables.
Japanese diplomacy leading to the Japan–Soviet Neutrality Pact is sometimes referred to as ‘Matsuoka diplomacy’, a style of foreign policy that directly reflected the personality of the Minister of Foreign Affairs, Yōsuke Matsuoka. 65 Matsuoka’s self-deceptive behaviour has been pointed out in historical studies and is considered a typical example of how self-deception influenced policymaking. As foreign minister in the second Fumimaro Konoe cabinet, Matsuoka presided over Japanese diplomacy from July 1940 to July 1945, a crucial period on the eve of the Pacific War. The greatest achievements of Matsuoka’s diplomacy were the conclusion of the Tripartite Pact (1940) and the Soviet-Japanese Neutrality Pact (1941), two treaties that ultimately pushed Japan towards the Pacific war. Matsuoka became the foreign minister with ambitions to become Prime Minister on 22 July 1940. 66 Since diplomacy had become subordinate to the military’s war leadership (as the second Sino-Japanese war showed), Konoe tried to regain control of diplomacy from the military by using Matsuoka’s leadership. 67 Matsuoka responded to Konoe’s expectations by controlling the military through power politics, thereby developing civilian-led diplomacy. 68 Therefore, Konoe virtually delegated the handling of foreign affairs to Matsuoka, who became the government’s representative regarding diplomacy. 69
Furthermore, Matsuoka successfully eliminated enemies within the Ministry of Foreign Affairs. In the 1930s, while the influence of the pro-American-British faction of the Kijūro Shidehara was receding, three groups calling for ‘innovative diplomacy’ (Kakushin Gaikō) had become the mainstream in the Ministry: the traditionalist group (Asian faction) of Hachirō Arita, the imperialist group of Toshio Shiratori and the power-balancing group of Matsuoka. 70 In this situation, Matsuoka eliminated the influence of Arita and Shiratori and centralised power within the Foreign Ministry through deft intra-ministerial politics. 71 By doing so, Matsuoka achieved a diplomatic mandate from Konoe, control of the military and elimination of adversaries within the Ministry, making the diplomatic process leading to the Soviet-Japanese Neutrality Pact almost entirely based on Matsuoka’s perceptions and preferences as follows.
In November 1939, Shigenori Tōgō (the Japanese ambassador in Moscow) persuaded Matsuoka to notice the deteriorating relationship between the USSR and Germany, stating ‘[r]ecently, the Soviet–Germany relationship has gone bad’. 72 Nevertheless, Tōgō’s advice did not affect Matsuoka’s judgement of the international context. In February 1941, Matsuoka received reports indicating the strained German-Soviet relationship, including one in which the German campaign against the Balkans made their relationships tense. 73 If Matsuoka was a rational actor, who strategically responds to systemic imperatives, he should have stopped visiting Europe when receiving such negative information by rationally updating the existing belief. Nevertheless, as suggested by the positive illusion hypothesis, Matsuoka confidently visited Europe, and according to Hosoya, ‘he was confident of his diplomatic manoeuvre to use most of Hitler and Stalin’. 74
When arriving in Berlin, the anti-Nazi coup occurred, and the Japan–Germany talks (March 27–30) provided Matsuoka with ‘the fact that there is no doubt of the deterioration of the Germany-Soviet relationship’. 75 However, he regarded the antagonistic behaviours of both states as ‘the bluff of each side’, expecting that ‘it will not lead to war’. 76 He was also confident that ‘if Germany could scare the USSR, showing an offensive attitude, the war will not necessarily occur’ (24 April). 77 Although Joachim von Ribbentrop (German Minister of Foreign Affairs) told Matsuoka about the gathering of many German soldiers at eastern Prussia’s border, Matsuoka flattered himself to ignore the implication of such gathering. 78 Considering a sequence of Matsuoka’s salient overconfidence, to illustrate the positive illusion hypothesis, diplomatic historian Nish aptly summarised Japan–Soviet Neutrality Pact as ‘basically a case of misjudgment by those who had an exaggerated confidence in their own perceptiveness’. 79
Meanwhile, Matsuoka encountered the unexpected hardline attitude of the USSR, forced to come to terms with the form of the neutrality pact, which he initially did not desire. 80 Nonetheless, why did he become confident in the suboptimal decision that ‘Japan, which strengthened the defence of the north, was feared to advance southward this time by Britain and the United States’ and that ‘the conclusion of the neutrality pact was highly important for Japan’? 81 If the information is constant, rationalist theories (e.g. offensive realism and rational choice theory) cannot explain this preference reversal because they basically assume that ‘preferences affect actions’, not vice versa. 82 In contrast to rationalist theories, the neoclassical realist model of overconfidence can explain this anomaly from the psychological perspective of self-deception, especially the cognitive dissonance reduction effect (hypothesis 2). When choosing a suboptimal option, humans usually feel psychological conflicts, such that ‘why did I make such a choice?’ or ‘was it better for me to take different options?’ To cope with the stressful situation, natural selection provides the brain with the psychological mechanism of self-deception, especially the cognitive dissonance reduction mechanism (i.e. reinterpreting the value of suboptimal options to resolve the psychological conflict stemming from the choice). This psychological mechanism helped Matsuoka to remain overconfident about his suboptimal decision (i.e. the neutrality pact).
Overall, Japanese foreign policy, from the four-power entente scheme to the Japan–Soviet Neutrality Pact, ‘owed much to Matsuoka’s overconfidence such that the world always revolves around him’. 83 This suboptimal balancing, which deviates from the dictates of the international system, resulted from ‘the excessively overconfident diplomacy of Matsuoka, whose practices sometimes leap the pursuit of aims, exceeding the objective limitations’. 84 Though such overconfidence is an anomaly of rationalist theories downplaying psychological factors, the neoclassical realist model of overconfidence can explain it from the perspective of self-deception (including the positive illusion and cognitive dissonance effect) without overlooking the structural constraint on state behaviours as the ‘first cut’ of the analysis.
What role, then, did intervening variables other than self-deception play in Japanese diplomacy leading to the Soviet-Japanese Neutrality Pact? Three factors deserve consideration: logrolling, nationalism and deficiencies in the Japanese political system (the irresponsible system). However, these variables are not significant enough to allow the self-deception thesis to be dismissed. First, according to the logrolling theory, 85 major decisions on foreign policy were made little by little, with the army and navy repeatedly competing and bargaining with each other, leading to the outbreak of the Pacific War. However, as mentioned above, Matsuoka, during this period, controlled the military to avoid the military-led policy and promoted civilian-led diplomacy. Therefore, historical evidence does not support the logrolling thesis concerning the Soviet-Japanese Neutrality Pact.
Second, according to the nationalism thesis, Japan’s fervent nationalism at the time was the cause of its belligerent foreign policy. 86 However, Japan’s hyper-nationalism, so-called ‘ultranationalism’, cannot explain the timing of the Neutrality Pact. The rise of nationalism in Japan occurred with the collapse of Taisho democracy, which was much earlier than in 1941. 87 That being the case, even if the hypothesis that ultranationalism leads to a belligerent foreign policy is correct, there is no necessity for Japan to conclude the Soviet-Japanese Neutrality Pact in 1941, and it would not be surprising if it had done so earlier. Thus, nationalism cannot explain the causal mechanism or timing of Japan’s conclusion of the Soviet-Japanese Neutrality Pact.
Third, the hypothesis that the deficiency of Japan’s domestic political system caused the Soviet-Japanese Neutrality Pact is also inconsistent with historical facts. The so-called ‘system of irresponsibility’ is often cited as a feature of Japanese foreign policy during the interwar period. 88 However, Japan’s political system lacked rulers who assumed ultimate political responsibility long before the Soviet-Japanese Neutrality Pact, especially when the Genro ceased having any political influence. 89 Furthermore, as mentioned earlier, Japanese diplomacy surrounding the Neutrality Pact was one in which political power was concentrated in Matsuoka personally, and the situation during this period was the exact opposite of the irresponsible system thesis.
Criticisms against the neoclassical realist model of overconfidence
Is neoclassical realism a degenerative research programme?
The first criticism is that neoclassical realism is a degenerative research programme attempting to salvage the realist research programme that has outlived its usefulness using domestic variables on an ad hoc basis. 90 For example, consider the development of the balance of power theory leading to neoclassical realism. Waltz’s neorealism holds that states balance against power. 91 However, it became apparent that states sometimes do not balance against power. To explain such anomalies, Walt developed the balance of threat theory, which holds that states balance against not only power but also threats. 92 Nonetheless, although the balance of threat theory can deal with external threats, it cannot theorise internal threats such as revolutions and coups. To fill the gap, David developed a realist version of the two-level game theory termed the omnibalancing theory, which holds that leaders balance against both external and internal threats. 93
After that, it became evident that ambiguity exists about the priority of threats (external or internal) that impose constraints on leaders’ choice of balancing strategies. To solve the problem, Lobell developed a neoclassical realist model termed the complex threat identification model, which regards external threats as independent variables, internal threats as intervening variables and state behaviours as dependent variables in a more explicit manner. 94 In light of the aforementioned development, existing criticisms against neoclassical realism that realists attempt to modify auxiliary assumptions (including power or threats) in an ad hoc manner or misuse domestic variables to save the flawed hardcore of the research programme may seem to be valid. Nevertheless, this conclusion is premature. In light of the MSRP, we can understand that neoclassical realism is not a degenerative research programme because it assumes the realist principles as the hardcore, adding domestic variables as auxiliary assumptions to develop the research programme further. 95 In other words, since neoclassical realists do not use domestic factors in an ad hoc manner (instead, they follow the procedures of the MSRP consciously or unconsciously), they do not fall into the degenerative research programme. 96
Unobservability of psychological factors
Second, how can it be argued that self-deception, essentially unobservable, exerts causal effects on balancing behaviours? What is the meta-theoretical ground for neoclassical realism’s use of psychological variables? Scientific realism has the advantage of answering this question because it enables scholars to theorise unobservable phenomena (e.g. biases and neurons) from pluralistic perspectives. 97 Scientific realism, which holds that entities postulated by mature scientific theories are believed by scientists to be real, is a crucial metatheory supported by leading IR scholars (such as Wight), 98 and among many theories of philosophy of science, it is considered especially useful in developing IR theories. Scientific realism aims to approach approximate truth about the world progressively, which exists independently of us, from the assumptions that (1) the unobservable treated in science indeed exists, even if researchers cannot directly experience them, and (2) scientific theories accepted in mature science are not absolutely but approximately true. 99 Using scientific realism as a metatheoretical foundation, IR theorists can defend the causal effects of psychological factors (i.e. self-deception) in a methodologically robust manner.
Among various theories of scientific realism, two theses are especially worth noting to defend the use of psychological factors in IR. The first is the detection (i.e. manipulation or intervention). 100 Even if researchers cannot directly observe objects, if they can access them by some tools/means (e.g. historical sources, experimental equipment, archaeological evidence and statistical data), then we can make a certain realistic assessment of their nature, progressively approaching the approximate truth of the world. It is what scientific realists understand by the concept of detection. Extending detection to international politics, we can understand that IR theorists can defend the causal effects of self-deception on leaders’ decisionmaking using relevant historical materials even if they cannot observe these phenomena in real-time.
The second is the inference to the best explanation (IBE). 101 IBE is the metatheoretical thesis resulting in the conclusion that (1) there is a novel phenomenon E that cannot be explained from what we know so far, (2) hypothesis H can explain the phenomenon E well, (3) there is currently no better hypothesis to explain the phenomenon E than hypothesis H and (4) hypothesis H is closer to the truth than any of other hypotheses. According to this reasoning, the validity of any given hypothesis is judged by how much better it is relative to other hypotheses. For example, even if we cannot empirically observe the brains of historical actors (including Matsuoka), if self-deception (i.e. positive illusion and cognitive dissonance effect) is the best explanation among various hypotheses for Japan’s suboptimal balancing, it is reasonable to attribute the cause to it.
What is the novelty of this case study?
The case study conducted to exemplify the neoclassical realist model of overconfidence does not attempt to uncover new historical facts but provides a novel theoretical explanation for a known phenomenon (i.e. Matsuoka’s overconfidence). If so, what is the significance of this type of case study? The answer lies in the use novelty in scientific realism, which refers to a metatheoretical criterion for theory evaluation developed by scientific realists, including Zahar and Worrall. 102 The important essence of the use novelty is that it is one thing to be aware of the existence of a phenomenon to be analysed and another to be able to give a novel theoretical explanation to it; moreover, theories can be constructed by predicting/explaining phenomena known at the time of its development.
It is widely known among IR theorists that leaders sometimes choose suboptimal balancing strategies driven by overconfidence. However, in the past, neoclassical realists have not provided robust scientific explanations (or foundations) for these facts from the perspective of self-deception. In contrast, this research provides a novel theoretical explanation from the standpoint of self-deception for the causal mechanisms by which self-deception leads to suboptimal balancing (i.e. positive illusion and cognitive dissonance reduction). Besides, it is one thing for historians to acknowledge that the Japan–Soviet Neutrality Pact results from the false optimism of Matsuoka, but it is quite another to provide novel theoretical explanations for the phenomenon in terms of self-deception, and doing the latter is the contribution (i.e. use novelty) of the neoclassical realist model of overconfidence.
Self-deception as an overarching cause of overconfidence
So far, this paper has argued that scientific realism has grounded theorists’ analysis of the unobservable factor, including self-deception. Finally, there supposes to be further criticism that Matsuoka’s overconfidence might be ascribed to other psychological factors beyond self-deception. These factors encompass a range of psychological variables such as personality and beliefs, learning experiences from previous successes, religious convictions and neuroscientific variables (e.g. alcohol consumption, increased testosterone). 103 For example, historians have described that Matsuoka was a markedly aggressive political figure in the Japanese government of this time. However, such criticism overlooks the unique role of self-deception as an all-encompassing cause underlying overconfidence-related factors. While numerous psychological and neuroscientific factors contribute to overconfidence, self-deception remains the fundamental cause unifying them, as systematically argued by Trivers. 104
Evolutionary psychology posits that what IR scholars label overconfidence is fundamentally rooted in self-deception. This perspective implies a deeply ingrained psychological mechanism prompting humans to disregard or deny evidence that challenges their beliefs or perceptions. At the heart of self-deception lies the inclination to reject the significance, relevance or weight of counterarguments and opposing evidence. Such denial can cultivate overconfidence: individuals deluding themselves into thinking they are correct – despite contrary evidence – are prone to unwarranted confidence in their beliefs.
Thus, self-deception is an encompassing notion that merges various factors related to overconfidence (e.g. anger and exclusive nationalism). 105 A deeper understanding of self-deception offers insights into these interconnected components and their influence on behaviour and decision-making. Trivers, a preeminent theorist on self-deception, suggests that individuals not only dismiss truths but also suppress distressing memories, fabricate false ones, justify unethical behaviours and act to enhance their self-view. 106 This array of ego-defence strategies is intricately linked with overconfidence. For instance, justifying unethical behaviours aimed at bolstering self-esteem can stem from an inflated belief in one’s own moral standing or superiority. The pervasive nature of self-deception indicates its role across a spectrum of decisions and behaviours, with overconfidence being the most famous manifestation. When individuals deceive themselves, they tend to base decisions on self-serving perceptions, potentially leading to overconfidence in diverse situations. Consequently, self-deception underlies overconfidence and associated elements.
Moreover, previous studies have revealed that self-deception interacts complementarily with relevant factors, including conservatism, aggression and testosterone. 107 Although the presence of any one of these factors may suggest that an actor is a strong self-deceiver, empirical evidence indicates that Matsuoka possessed multiple personality traits typically associated with potent self-deception, such as aggressiveness, a desire for power and overconfidence. Thus, it is theoretically sound to conclude that Matsuoka was indeed a bearer of profound self-deception.
Conclusion
Why did Japan conclude the Japan–Soviet Neutrality Pact in 1941 even though it knew that German-Soviet relations were deteriorating and heading towards a catastrophic war? When Japan and Germany had signed a military alliance, it would be unreasonable for Japan to approach the USSR, which Germany had determined to attack. This article examined the longstanding historical puzzle of Japan’s conclusion of Japan–Soviet Neutrality Pact by developing a novel neoclassical realist model termed ‘neoclassical realist model of overconfidence’ based on self-deception in evolutionary psychology using scientific realism as metatheoretical foundations. Drawing on neoclassical realism, this article argued that Japan’s balancing strategy during 1940–41, which initially reflected the tripolar structure of the international system (independent variable), ultimately resulted in the suboptimal balancing of the Japan–Soviet Neutrality Pact (dependent variable) because of a sequence of Matsuoka’s self-deception (intervening variable). This paper clarified the causal processes by which Matsuoka continued to adhere to the approach towards the USSR even though the initial strategic rationale for the rapprochement (i.e. the ultimate realisation of the four-power entente) had been lost due to the deterioration of German-Soviet relations leading to the disastrous war.
Driven by self-deception, Matsuoka underestimated the information about the exacerbated relations between the USSR and Germany, standing behind the idea of rapprochement with the former. Consequently, he chose the suboptimal balancing strategy of the Japan–Soviet Neutrality Pact. First, according to the positive illusion hypothesis, Matsuoka was overconfident about his bargaining power against the USSR, German pressure towards the USSR and the possibility of averting the German-Soviet war, while underestimating the bargaining power of the USSR against Japan, the risk of the deterioration of the Japan-Germany and Japan-U.S. relationships caused by the rapprochement with the USSR, and the possibility of the German-Soviet war. Second, according to the cognitive dissonance hypothesis, although Matsuoka desired the four-power entente (i.e. a stronger alliance with the USSR than the neutrality pact), he was forced to conclude the neutrality pact. To deal with the psychologically painful situation, he remained overconfident by positively reinterpreting the strategic value of the neutrality pact.
This paper includes several implications for decisionmakers and IR. First, as a policy implication, the neoclassical realist model of overconfidence warns leaders of the pitfalls of psychological biases (including self-deception), encouraging them to develop new decisionmaking structures (e.g. devil’s advocates to challenge prevailing wisdom) to deal with them. It also reminds us that leaders need to be aware of the problems resulting from misperception in general (not only self-deception but also other various biases), which leads to poor strategies. 108 Of course, leaders need to recognise that adversaries may face the same problems, regardless of whether they are aware of them. Assuming that opponents are rational actors, who perceive information objectively and act strategically, balancing strategies will be flawed because of the erroneous assumption. Thus, leaders need to consider their adversaries’ psychological states more carefully.
Second, it develops IR research on overconfidence from the novel perspective of self-deception without falling into the trap of reductionism. Until now, scholars have viewed the problem of leaders’ overconfidence causing foreign policy failures as an important research puzzle to be solved. 109 However, previous research has been unable to theorise overconfidence with adequate scientific foundations because they ignore self-deception, which is the crucial cause of overconfidence, failing to address the level of analysis problem consciously (i.e. lack of neoclassical realist model on overconfidence). This article aims to fill the gap by constructing a new neoclassical realist model, which explains suboptimal balancing behaviours driven by overconfidence, by introducing self-deception in evolutionary psychology (a domestic factor as an intervening variable) into the realist structural logic (systemic factors as independent variables).
Finally, this paper has the following limitations. Theoretically, the neoclassical realist model of overconfidence remains underdeveloped and requires more detailed causal mechanisms and theorising regarding its hypotheses (i.e. positive illusion hypothesis and cognitive dissonance hypothesis). Especially, more in-depth theoretical consideration is needed on the consequence of the interaction between self-deception (domestic variable) and the international system (structural variable). According to the latest finding of neoclassical realism, since ‘over time grand strategic choices can affect relative power and international outcomes, at times reshap[ing] the structure of the international system, neoclassical realism can also contribute to an explanation of changes in the nature of the international system’. 110 In light of this neoclassical realist insight, for instance, it could be said that the outbreak of the German-Soviet war in 1941 (i.e. a system-level phenomenon) was indirectly driven by mutual self-deception between Hitler and Stalin (i.e. unit-level variables). 111
Empirically, more cases other than the Japan–Soviet Neutrality Pact may help verify the neoclassical realist model of overconfidence. For example, Freedman argues that Putin’s self-deception is responsible for the current Russian-Ukrainian war as with Lyndon Johnson in the Vietnam War. 112 Although it is one example, future studies need to delve deeper into the neoclassical realist model of overconfidence by examining leaders regarded as typical examples of self-deception. Since this article’s case study is a plausibility probe, which does not aim to test or generalise models but merely illustrates them to show that the argument is sufficiently grounded in the evidence to justify further research, 113 the next step is more detailed case studies to test the logic of the model.
Footnotes
Declaration of conflicting interests
The author declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author received no financial support for the research, authorship, and/or publication of this article.
