Abstract
Previously in this journal, Roskes, Sligte, Shalvi, and De Dreu (2011) claimed to have uncovered a stronger tendency for soccer goalkeepers to dive to the right than to the left during penalty shoot-outs when their team is behind (a situation that Roskes et al. argue elicits an approach motivation). They interpret this finding as suggesting that “approach motivation, a focus on achieving positive outcomes, is related to relative left-hemispheric brain activation, which translates to a variety of right-oriented behavioral biases” (p. 1403). These results are also relevant to economics, in which game theory suggests that a rational goalkeeping strategy involves randomly choosing which way to dive so as to equalize the probability of scoring regardless of the direction the kicker chooses, a finding confirmed in penalty-kick data by Chiappori, Levitt, and Groseclose (2002).
We ran three analyses that cast doubt on the inferences Roskes et al. drew from their penalty-kick data. First, we found no statistically significant bias for diving right versus left when we replicated Roskes et al.’s study by collecting data on a new sample of penalty kicks. Second, we showed that the documented behavior is not a dysfunctional bias, but rather that this tendency is consistent with goalkeepers following a rational strategy in diving right in these situations, given that this is the direction in which kickers are more likely to kick. And third, we showed that Roskes et al.’s results are not robust to an alternative, arguably more natural coding of the game situations likely to elicit approach motivation. We have no quibble with the separate findings the authors present from a line-bisection task.
Replication Using a Separate Data Set
In order to independently replicate Roskes et al.’s findings, we constructed a new data set of penalty-kick shoot-outs by analyzing the online archives of games played in the Copa América tournament and the Union of European Football Associations (UEFA) Champions League during the years 1984 to 2011. This yielded a sample of 240 attempts, which provided a natural complement to the 204 attempts analyzed in Roskes et al.’s study of World Cup shoot-outs from 1982 to 2010. We coded all of the measures in the same way as they did in order to accurately replicate their study. As in Roskes et al.’s study, each penalty kick was coded by three independent observers. Coders agreed on 93% of the penalty kicks, and agreement was reached on the remaining 7% following repeated viewing of those penalties.
In Table 1, we provide the main results using both Roskes et al.’s data and the new data that we collected. The earlier study revealed a greater propensity for the goalkeepers to dive to the right when their team was behind (70.8%) versus when their team was tied or ahead (49.0% and 46.3%, respectively). Our new data show a smaller difference based on the game situation (65.6% when their team was behind, 56.0% when tied, 55.6% when ahead), and none of these percentages are statistically significantly different from each other.
Goalie and Kicker Behavior Across Different Game Situations
Note: The data from Roskes et al. (2011) are based on all Fédération Internationale de Football Association (FIFA) World Cup shoot-outs between 1982 and 2010. The new data we collected are based on all shoot-outs during the Copa América tournament and the Union of European Football Associations (UEFA) Champions League final games between 1984 and 2011.
The Goalie’s Strategy When Behind
Is the observed propensity of goalkeepers to dive right a dysfunctional strategy due to a directional bias, or is it simply a rational soccer strategy given the behavior of kickers? Roskes et al. argue that it is the former, noting that the kicker is more likely to score a goal when the goalie’s team is behind, relative to when his or her team is not. However, this is not evidence of a dysfunctional strategy, as it involves a comparison of two states of the game (whether or not your team is behind, which the goalkeeper does not choose), rather than comparing the results from the two strategic choices the goalkeeper actually faces (dive left or right). The problem here is not only that this is not the choice the goalkeeper faces, but it also confounds the success of different strategies with the effects that different states (being ahead or behind) may have on the kicker or goalkeeper. Instead, the appropriate test is to ask whether or not, for any particular state of the game, an alternative strategy of diving left more often than right would have yielded better results. That is, for the data in Table 1, Roskes et al. compared entries in the first column of each block, whereas the appropriate test compares the final two entries in each row, which represent the goalkeeper’s possible choices.
In Roskes et al.’s data, there are 24 penalty kicks that occurred when the goalie’s team was behind. Diving left never led to good outcomes, resulting in the opposite team’s making a goal 100% of the time, while diving to the right was a slightly more successful strategy, resulting in the opposition’s making a goal 86.7% of the time. In our newly collected data, the corresponding percentages were 100% and 85.7%. We also constructed an alternative measure of success, asking whether the goalie dives in the correct direction (irrespective of whether a goal results or the shot goes wide). Using Roskes et al.’s data, we found that this occurred 41.2% of the time that the goalie dived to the right and only 14.3% of the time that the goalie dived to the left, and in our new sample, the corresponding percentages were 46.0% and 37.5%, respectively.
Thus for neither sample nor either measure of success is there evidence of a dysfunctional bias in which it would have made sense for the goalkeeper to dive to the right less often than to the left. The reason that diving right in these situations is a good choice is that the kicker is more likely to kick to the goalkeeper’s right: 54.2% of the time versus 37.5%. To be clear, the point is not that goalkeepers should dive right more often—the difference in goals saved is not statistically significant—but rather that there is no evidence for Roskes et al.’s claims that goalkeepers are jumping to the right more often than if they were making rational strategic decisions.
Coding the Game Situation
Consider the 1990 game between Ireland and Romania. Both teams made their first four penalty shots, and then the Romanian goalie saved Ireland’s fifth (and last) attempt. At that point, the only way for Ireland to win the game was for the Irish goalkeeper to save Romania’s last shot, which would force the game to go to extra penalties. Because the Irish goalkeeper surely perceived his team to be behind, this situation provides data relevant to Roskes et al.’s claim that “approach motivation should be even stronger for goalkeepers whose team is behind and whose role in regaining the possibility to win the game is crucial” (p. 1405).
Yet Roskes et al. miscoded this as an instance in which approach motivation would not be strong. The problem is simply that they looked to see only whether a team was behind in terms of goals scored at the time the penalty shot was taken (in this example, the score was 4–4) without accounting for the number of shots missed (1 for Ireland, 0 for Romania).
Thus, we provide an alternative—and arguably preferable—approach in which a team is coded as behind if it has more shots missed. Given that “failures to score are rare” (Roskes et al., 2011, p. 1405), this seems more intuitive, and it is in accord with the soccer fan’s intuition that if Ireland had won, that would have been a come-from-behind victory. Moreover, this more faithfully represents Roskes et al.’s statement that they intended to capture situations “when a penalty taker from the goalkeeper’s team has missed a previous penalty” (p. 1405).
This recoding roughly doubled the number of situations in which goalkeepers perceived themselves to be behind and hence subject to approach motivation. In Table 1, we report the numbers based on the revised classification of game situations using this alternative coding. While the original classification indicated that goalies dived to the right 70.8% of the time when their teams were behind, this number dropped to 56.6% when we used the alternative classification and was no longer statistically different from what occurred when the team was not behind. Furthermore, in our new replication data set, we found no significant difference in goalies’ tendency to dive right across game states under either definition of “behind” (using the revised coding, it was 62.5% vs. 54.8%). Using our preferred definition of situations likely to elicit approach motivation, we found no significant link between such motivation and goalkeepers’ tendency to dive right in either data set. At a minimum, the conclusion of Roskes et al. is highly sensitive to the assumption about which situations are likely to yield approach motivation.
On the basis of these three analyses, we found little evidence that soccer goalies exhibit a right-oriented bias when they are behind at the end of the game. Clarifying this point is important because the claims made in Roskes et al.’s article suggest that the predictions of game theory about mixed-strategy equilibrium were violated, and these are central to decision making in a broad set of situations.
Footnotes
Declaration of Conflicting Interests
The authors declared that they had no conflicts of interest with respect to their authorship or the publication of this article.
Open Practices
All data have been made publicly available via Open Science Framework and can be accessed at https://osf.io/q6gbw/. The complete Open Practices Disclosure for this article can be found at http://pss.sagepub.com/content/by/supplemental-data. This article has received the badge for Open Data. More information about the Open Practices badges can be found at https://osf.io/tvyxz/wiki/view/ and 
.
References
Supplementary Material
Please find the following supplemental material available below.
For Open Access articles published under a Creative Commons License, all supplemental material carries the same license as the article it is associated with.
For non-Open Access articles published, all supplemental material carries a non-exclusive license, and permission requests for re-use of supplemental material or any part of supplemental material shall be sent directly to the copyright owner as specified in the copyright notice associated with the article.
