Quantifying the unfairness of the 2018 FIFA World Cup qualification

Abstract

This paper investigates the fairness of the 2018 FIFA World Cup qualifying competition via Monte Carlo simulations. The qualifying probabilities are calculated for 102 nations, all teams except for African and European countries. A method is proposed to quantify the degree of unfairness. Although the qualifications within four FIFA confederations are constructed fairly, serious differences are found between the continents: for instance, a South American team could have tripled its chances by playing in Asia. Choosing a fixed matchup in the inter-continental play-offs instead of the current random draw can reduce the unfairness of the competition. The move of Australia from the Oceanian to the Asian zone is shown to increase its probability of participating in the 2018 FIFA World Cup by about 65%. Our results provide important insights for the administrators on how to reallocate the qualifying berths.

Keywords

Association football,ethics,soccer,tournament design

Aber so viel ist an sich klar, daßdieser, wie jeder Gegenstand, der unser Begreifungsvermögen nicht übersteigt, durch einen untersuchenden Geist aufgehellt und in seinem inneren Zusammenhang mehr oder weniger deutlich gemacht werden kann, und das allein reicht schon hin, den Begriff der Theorie zu verwirklichen.¹

(Carl von Clausewitz: Vom Kriege)

Introduction

The FIFA World Cup, the most prestigious soccer tournament around the world, is followed by millions of fans. According to Palacios-Huerta,¹ 5% of all the people who ever lived on the Earth watched the final of the 2010 FIFA World Cup played by the Netherlands and Spain. Qualification to the FIFA World Cup creates widespread media coverage in the competing countries² and brings significant economic benefits:³ each participating team has received at least 9.5 million USD in the 2018 FIFA World Cup.⁴ Success in soccer can even help build nations.⁵

Obviously, every national team cannot play in the FIFA World Cup. Thus there is a World Cup qualification competition, consisting of a series of tournaments organised by the six FIFA confederations. In 2006, Australia left the Oceania Football Confederation (OFC) to join the Asian Football Confederation (AFC). Since then, the country qualified for all FIFA World Cups (2010, 2014, 2018), although it played only twice before (1974, 2006). Is this a mere coincidence, the result of a fruitful team development strategy, or can be partially explained by the change of affiliation? The current paper discusses the third issue by analysing the fairness of the 2018 FIFA World Cup qualification.

Some scientific research has addressed the World Cup qualifiers. Flégl⁶ applies Data Envelopment Analysis (DEA) to evaluate the performance of national soccer teams during the 2014 FIFA World Cup qualification. Stone and Rod³ aim to assess the degree to which the allocation of qualification berths among the six FIFA confederations reflects the quality of the teams from their specific region, mainly by descriptive statistics. Durán et al.⁷ recommend integer programming to construct alternative schedules for the South American qualifiers that overcome the main drawbacks of the previous approach. Their proposal has been unanimously approved by the South American associations to use in the 2018 FIFA World Cup qualification. Pollard and Armatas⁸ investigate home advantage in the FIFA World Cup qualification games. Csató⁹ identifies an incentive incompatibility problem in the European section of the recent FIFA World Cup qualifications.

However, the FIFA World Cup qualification competition has never been studied before with respect to the chances of the participating teams. A possible reason is the complexity of the qualifying system as will be seen later.

Our paper aims to fill this research gap. In particular, the probability of qualification to the 2018 FIFA World Cup is quantified for the 102 nations of the AFC, CONCACAF (Confederation of North, Central American and Caribbean Association Football), CONMEBOL (South American Football Confederation), and OFC to answer three questions: (a) Is the qualification process designed fairly in the sense that it provides a higher chance for a better team both within and between the confederations? (b) Is it possible to improve fairness without reallocating the qualifying berths? (c) How did the move of Australia from the OFC to the AFC in 2006 affect the teams?

The main contributions can be summarised as follows:

First in the academic literature, the paper calculates the qualifying probabilities for the FIFA World Cup based on the Elo ratings of the national teams.

A method is proposed to measure the degree of unfairness. It shows that all the four qualifiers are constructed fairly. On the contrary, substantial differences are found between the confederations.

Using a well-devised fixed matchup in the inter-continental play-offs—a policy applied in the 2010 FIFA World Cup qualification—instead of the current random draw can reduce unfairness by 7.5%.

Australia has increased its probability of playing in the 2018 FIFA World Cup by 65% as a result of leaving the OFC and joining the AFC. The move has been detrimental to any AFC nation, while it has favoured all other countries, especially New Zealand. The change has reduced unfairness by almost 20%, which might explain why the move of Australia was approved.

Our approach can be applied in any sports where teams contest in parallel tournaments for the same prize, hence the natural issue of equal treatment of equals emerges. While these designs have recently been analysed with respect to incentive incompatibility^10,11,12, the numerical investigation of fairness is currently limited to the qualification for the UEFA Euro 2020.¹³ Chapter 6.4.

Related literature

The paper contributes to three different fields of research: fairness in sports, analysis of FIFA competitions and rules, and simulation of tournament designs.

Fairness in sports is a widely discussed topic in the literature. From the perspective of tournament design, there are two crucial axioms: (1) stronger players should be preferred to weaker players; and (2) equal players should be treated equally. Otherwise, an incentive might exist to manipulate the tournament. Groh et al.¹⁴ check which seedings in elimination tournaments satisfy the two properties. Arlegi and Dimitrov¹⁵ apply these requirements to different kinds of knockout contests and characterise the appropriate structures. Csató¹³ Chapter 6.4 shows the unfairness of the qualification for the 2020 UEFA European Championship with important lessons for sports management.¹⁶ Both theoretical^17–20 and empirical²¹ investigations reveal that the ex-ante winning probabilities in round-robin tournaments with three and four symmetric players may depend on the schedule, which can lead to severe problems in the 2026 FIFA World Cup.²² Soccer penalty shootouts seem to be advantageous for the first shooter^1,23,24 but this bias can be mitigated by a carefully devised mechanism.^25–30 The knockout bracket of the 2016 UEFA European Championship has created imbalances among the six round-robin groups of four teams each.³¹ Vaziri et al.³² state some fairness properties of sports ranking methods.

In contrast to the World Cup qualifiers, several attempts have been made to forecast the FIFA World Cup final tournament games. Dyte and Clarke³³ treat the goals scored by the teams as independent Poisson variables to simulate the 1998 FIFA World Cup. Deutsch³⁴ aims to judge the impact of the draw in the 2010 World Cup, as well as to look back and identify surprises, disappointments, and upsets. Groll et al.³⁵ fit and examine two models to forecast the 2014 FIFA World Cup. O’Leary³⁶ finds that the Yahoo crowd was statistically significantly better at predicting the outcomes of matches in the 2014 World Cup compared to the experts and was similar in performance to established betting odds. Groll et al.³⁷ propose a novel hybrid modelling approach by combining random forests with Poisson ranking methods to predict the 2018 FIFA World Cup. This technique has also been used to forecast the 2019 FIFA Women’s World Cup³⁸ and the 2020 UEFA European Championship.³⁹

Further aspects of the FIFA World Cup have also been researched extensively. Jones⁴⁰ and Rathgeber and Rathgeber⁴¹ discuss the consequences of the unevenly distributed draw procedures for the 1990 and 2006 FIFA World Cups, respectively. Scarf and Yusof⁴² reveal the effect of seeding policy and other design changes on the progression of competitors in the World Cup final tournament. Guyon⁴³ collects some flaws and criticisms of the World Cup seeding system. Laliena and Lopez⁴⁴ and Cea et al.⁴⁵ provide a detailed analysis of group assignment in the FIFA World Cup.

Finally, since historical data usually do not make it possible to calculate the majority of tournament metrics such as qualifying probabilities, it is necessary to use computer simulations for this purpose, especially for evaluating new designs.⁴⁶ Any simulation model should be based on a prediction model for individual ties. According to Lasek et al.,⁴⁷ the best performing algorithm of ranking systems in soccer with respect to accuracy is a version of the famous Elo rating. Baker and McHale⁴⁸ provide time-varying ratings for international soccer teams. Van Eetvelde and Ley⁴⁹ overview the most common ranking methods in soccer. Ley et al.⁵⁰ build a ranking reflecting the teams’ current strengths and illustrate its usefulness by examples where the existing rankings fail to provide enough information or lead to peculiar results. Corona et al.⁵¹ propose a Bayesian approach to take into account the uncertainty of parameter estimates in the underlying match-level forecasting model.

The 2018 FIFA World Cup qualification

The FIFA World Cup qualification is a series of tournaments to determine the participants of the FIFA World Cup. Since 1998, the final competition contains 32 teams such that the host nation(s) receive(s) a guaranteed slot. The number of qualifying berths for the continents is fixed from 2006 to 2022 as follows:

AFC (Asian Football Confederation): 4.5;

CAF (Confederation of African Football): 5;

CONCACAF (Confederation of North, Central American and Caribbean Association Football): 3.5;

CONMEBOL (South American Football Confederation): 4.5;

OFC (Oceania Football Confederation): 0.5;

UEFA (Union of European Football Associations): 13.

The six confederations organise their own contests. The 0.5 slots represent a place in the inter-continental play-offs, which is the only interaction between the qualifying tournaments of different geographical zones.

The qualifications of all confederations are played in rounds. Each round is designed either in a knockout format (where two teams play two-legged home-away matches) or in a round-robin format (where more than two teams play in a single or home-away group against every other team of the group). The rounds are often seeded, that is, the participating countries are divided into the same number of pots as the number of teams per group (meaning two pots in the knockout format) and one team from each pot goes to a given group. The traditional seeding is based on an exogenously given ranking—usually the FIFA World Ranking at a specific date—such that, if a pot contains $k$ teams, the best $k$ teams are in the first pot, the next $k$ are in the second pot, and so on.

Our paper focuses on four qualifications, the AFC, the CONCACAF, the CONMEBOL, and the OFC because (1) contrary to the CAF and UEFA competitions, they are connected to each other; (2) the largest and most successful nation of the OFC, Australia, switched to the AFC in 2006.

The 2018 FIFA World Cup qualification (AFC) contained $46$ nations and four rounds. The starting access list was determined by the FIFA World Ranking of January 2015.

First round

Format: knockout

Competitors: the $12$ lowest-ranked teams ( $35$ – $46$ )

Seeding: traditional; based on the FIFA World Ranking of January 2015

Second round

Format: home-away round-robin, 8 groups of five teams each

Competitors: the $34$ highest-ranked teams ( $1$ – $34$ ) + the six winners from the first round

Seeding: traditional; based on the FIFA World Ranking of April 2015²

Third round

Format: home-away round-robin, 2 groups of six teams each

Competitors: the eight group winners and the four best runners-up in the second round³

Seeding: traditional; based on the FIFA World Ranking of April 2016

The two group winners and the two runners-up qualified to the 2018 FIFA World Cup.

Fourth round

Format: knockout

Competitors: the third-placed teams from the groups in the third round

Seeding: redundant

The winner advanced to the inter-confederation play-offs.

The 2018 FIFA World Cup qualification (CONCACAF) contained

35

nations and five rounds. The access list was determined by the FIFA World Ranking of August 2014.

First round

Format: knockout

Competitors: the $14$ lowest-ranked teams ( $22$ – $35$ )

Seeding: traditional; based on the FIFA World Ranking of August 2014

Second round

Format: knockout

Competitors: the teams ranked $9$ – $21$ in the access list + the seven winners from the first round

Seeding: the seven teams of pot 5 (ranked $9$ – $15$ ) were drawn against the teams of pot 6 (the winners from the first round) and the three teams of pot 3 (ranked $16$ – $18$ ) were drawn against the three teams of pot 4 (ranked $19$ – $21$ ); based on the FIFA World Ranking of August 2014

Third round

Format: knockout

Competitors: the teams ranked $7$ – $8$ in the access list + the $10$ winners from the second round

Seeding: traditional; based on the FIFA World Ranking of August 2014

Fourth round

Format: home-away round-robin, 3 groups of four teams each

Competitors: the teams ranked $1$ – $6$ in the access list + the six winners from the third round

Seeding: pot 1 (teams ranked $1$ – $3$ ), pot 2 (teams ranked $4$ – $6$ ), pot 3 (the winners from the third round) such that each group contained a team from pot 1, a team from pot 2, and two teams from pot 3; based on the FIFA World Ranking of August 2014

Fifth round

Format: home-away round-robin, one group of six teams

Competitors: the group winners and the runners-up in the fourth round

Seeding: redundant

The top three teams qualified to the 2018 FIFA World Cup and the fourth-placed team advanced to the https://en.wikipedia.org/wiki/2018˙FIFA˙World˙Cup˙qualification˙(inter-confederation˙play-offs)inter-confederation play-offs.

The CONCACAF competition can be criticised for the great role attributed to the FIFA World Ranking. Perhaps it is not only a coincidence that this confederation has planned to fundamentally restructure its qualifying tournament for the 2022 FIFA World Cup⁵².

The 2018 FIFA World Cup qualification (CONMEBOL) contained $10$ nations, which contested in a home-away round-robin tournament⁷. The top four teams qualified to the 2018 FIFA World Cup, and the fifth-placed team advanced to the inter-confederation play-offs.

The 2018 FIFA World Cup qualification (OFC) contained $11$ nations and four rounds.

First round

Format: single round-robin, one group organised in a country (Tonga was chosen later)

Competitors: the four lowest-ranked teams ( $8$ – $11$ ), based on FIFA World Ranking and sporting reasons

Seeding: redundant

Second round

Format: single round-robin, 2 groups of four teams each, all matches played in one country

Competitors: the seven strongest teams ( $1$ – $7$ ) + the group winner in the first round

Seeding: traditional; based on the FIFA World Ranking of July 2015

Third round

Format: home-away round-robin, 2 groups of three teams each

Competitors: the top three teams from each group in the second round

Seeding: pot 1 (2016 OFC Nations Cup finalists), pot 2 (2016 OFC Nations Cup semifinalists), pot 3 (third-placed teams in the second round) such that each group contained one team from pots $1$ – $3$ each⁴

Fourth round

Format: knockout

Competitors: the group winners in the third round

Seeding: redundant

The winner advanced to the inter-confederation play-offs.

Consequently, the inter-confederation play-offs were contested by four teams from the four confederations (AFC, CONCACAF, CONMEBOL, OFC), and were played in a knockout format. The four nations were drawn randomly into two pairs without seeding. The two winners qualified to the 2018 FIFA World Cup. The inter-confederation play-offs of the 2006 FIFA World Cup and the 2014 FIFA World Cup qualification were also drawn randomly. This policy will be followed in the 2022 FIFA World Cup, too. However, FIFA fixed the ties in the inter-continental play-offs of the 2010 FIFA World Cup qualification as AFC vs. OFC and CONCACAF vs. CONMEBOL to pair teams being in closer time zones.⁵

Methodology and implementation

This section presents how the outcome of a qualifying match is determined, the simulation is carried out, and the unfairness of a qualification tournament is measured.

The simulation model

In order to quantify a particular tournament metric of the FIFA World Cup qualification, it is necessary to follow a simulation technique because historical data are limited: the national teams do not play many matches and the outcome of a qualification is only a single realisation of several random variables. Such a model should be based on predicting the result of individual games. For this purpose, the strengths of the teams are measured by the World Football Elo Ratings, available at the website http://eloratings.net/eloratings.net. Elo-inspired methods are usually good in forecasting⁴⁷, and have been widely used in academic research.^45,53,54

Elo ratings depend on the results of previous matches but the same result is worth more when the opponent is stronger. Furthermore, playing new games decreases the weight of earlier matches. Since there is no official Elo rating for national teams, this approach can be implemented in various ways. For instance, while the official https://en.wikipedia.org/wiki/FIFA˙World˙RankingsFIFA World Ranking adopted the Elo method of calculation after the 2018 FIFA World Cup,^55,56 it does not contain any adjustment for home or away games. However, home advantage has been presented to be a crucial factor in international soccer, even though its influence appears to be narrowing.⁴⁸ The World Football Elo Ratings takes into account some soccer-specific parameters such as the margin of victory, home advantage, and the tournament where the match was played.

In the 2018 FIFA World Cup qualification, three types of matches were played: group matches in a home-away format, single group matches in a randomly chosen country that is assumed to be a neutral field (only in the first and the second rounds in the OFC zone), and home-away knockout matches. The win expectancy of team $i$ against team $j$ can be directly obtained from the formula of Elo rating according to the system of World Football Elo Ratings (see http://eloratings.net/about):

W_{i j} = \frac{1}{1 + 10^{- (E_{i} - E_{j}) / 400}},

(1)

where

E_{i}

and

E_{j}

are the Elo ratings of the two teams, respectively, such that the rating of the home team is increased by

100

In knockout clashes, the teams focus primarily on advancing to the next round rather than winning one match. Therefore, we have followed the solution of the ClubElo rating (see http://clubelo.com/System), namely, these two-legged matches are considered as one long match with a corresponding increase in the difference between the strengths of the teams. The probability that team $i$ advances against team $j$ is:

W_{i j}^{(K O)} = \frac{1}{1 + 10^{- \sqrt{2} (E_{i} - E_{j}) / 400}} .

(2)

In the case of group matches, the number of goals scored by each team in each match should also be modelled because drawn matches are common in football, and ties between teams with the same score are decided by goal difference. For this purpose, the number of goals is assumed to follow a Poisson distribution, which is a standard solution in the literature, see, e.g. Chater et al.⁵⁷ or Dagaev and Rudyak.⁵⁸ In particular, team

i

scores

k

goals against team

j

with the probability

P_{i j} (k) = \frac{{(λ_{i j}^{(f)})}^{k} \exp (- λ_{i j})}{k!},

(3)

where the parameter

λ_{i j}^{(f)}

is the expected number of goals scored by team

i

against team

j

if the match is played on field

f

(home:

f = h

; away:

f = a

; neutral

f = n

The value of parameter $λ_{i j}^{(f)}$ in (3) is modelled as a quartic polynomial of $W_{i j}$ (see (1)) estimated using a least squares regression with a regime change at $W_{i j} = 0.9$ .⁵⁹ On the basis of more than 29 thousand home-away matches played by national football teams, the average expected number of goals for the home team $i$ is

λ_{i j}^{(h)} = {\begin{matrix} - 5.42301 \cdot W_{i j}^{4} \\ + 15.49728 \cdot W_{i j}^{3} \\ - 12.6499 \cdot W_{i j}^{2} \\ + 5.36198 \cdot W_{i j} \\ + 0.22862 & if W_{i j} \leq 0.9 \\ 231098.16153 \cdot (W_{i j} - 0.9)^{4} \\ - 30953.10199 \cdot (W_{i j} - 0.9)^{3} \\ + 1347.51495 \cdot (W_{i j} - 0.9)^{2} \\ - 1.63074 \cdot (W_{i j} - 0.9) \\ + 2.54747 & if W_{i j} > 0.9 \end{matrix}

(4)

with

R^{2} = 0.984

, whereas the average number of goals for the away team

j

equals

λ_{i j}^{(a)} = {\begin{matrix} 90173.57949 \cdot (W_{i j} - 0.1)^{4} \\ + 10064.38612 \cdot (W_{i j} - 0.1)^{3} \\ + 218.6628 \cdot (W_{i j} - 0.1)^{2} \\ - 11.06198 \cdot (W_{i j} - 0.1) \\ + 2.28291 & if W_{i j} < 0.1 \\ - 1.25010 \cdot W_{i j}^{4} \\ - 1.99984 \cdot W_{i j}^{3} \\ + 6.54946 \cdot W_{i j}^{2} \\ - 5.83979 \cdot W_{i j} \\ + 2.80352 & if W_{i j} \geq 0.1 \end{matrix}

(5)

with

R^{2} = 0.955

In the OFC zone, some matches are played on neutral ground when, based on almost 10 thousand matches, the average number of goals for team $i$ is

λ_{i j}^{(n)} = {\begin{matrix} 3.90388 \cdot W_{i j}^{4} \\ - 0.58486 \cdot W_{i j}^{3} \\ - 2.98315 \cdot W_{i j}^{2} \\ + 3.13160 \cdot W_{i j} \\ + 0.33193 & if W_{i j} \leq 0.9 \\ 308097.45501 \cdot (W_{i j} - 0.9)^{4} \\ - 42803.04696 \cdot (W_{i j} - 0.9)^{3} \\ + 2116.35304 \cdot (W_{i j} - 0.9)^{2} \\ - 9.61869 \cdot (W_{i j} - 0.9) \\ + 2.86899 & if W_{i j} > 0.9 \end{matrix}

(6)

with

R^{2} = 0.976

The ranking of the teams in a group is determined as follows: (a) greatest number of points obtained in all group matches; (b) goal difference in all group matches; (c) greatest number of goals scored in all group matches; (d) drawing of lots.

The Elo ratings are dynamic but we have fixed them for the sake of simplicity. In each of the four confederations, the ratings of all teams on the day before the first match of the relevant qualification tournament are considered. Four tables in the Appendix show the corresponding measures of strength: Table A.1 for the $35$ CONCACAF teams; Table A.2 for the $46$ AFC teams; Table A.3 for the $10$ CONMEBOL teams; and Table A.4 for the $11$ OFC teams.

Any theoretical model is only as good as its assumptions. Therefore, it is worth summarising the main limitations:

The strength of the teams is exogenously given and fixed during the whole qualification process.

Home advantage does not differ between the confederations despite the findings of Pollard and Armatas⁸. However, the influence of the corresponding parameter is minimal since all matches are played both home and away except for Oceania, where some games are hosted by a randomly drawn country.

The schedule of the matches is not accounted for.

The efforts of the teams do not change even if they have already qualified or have been eliminated.

These shortcomings mean that the numerical results are primarily for comparative purposes.

Implementation

Our computer code closely follows the rules of the qualification process described in the previous section. Pot assignment and seeding are based on the Elo rating of the teams in each case since the strengths of the teams are given by this measure instead of the FIFA ranking, which was a rather bad predictor before the 2018 reform^45,47. Although the official AFC qualification updated the ranking of the teams before the seeding in each round—thus the results of matches played already during the qualification may have affected the subsequent rounds—, that complication is disregarded in our simulations. Using the Elo rating for seeding is also necessary to guarantee the consistency of the simulations since the proposed fairness measure will be based on the Elo ratings. Therefore, every instance of unfairness will be intrinsic to the tournament design.

Finally, the move of Australia from the OFC to the AFC will also be evaluated. Accordingly, an alternative design of the FIFA World Cup qualification should be chosen with Australia being in the OFC instead of the AFC. Since then there are only $45$ countries in Asia, a straightforward format would be to organise the first knockout round with the 10 lowest-ranked teams ( $36$ – $45$ ), while the second round is contested by the $35$ highest-ranked teams ( $1$ – $35$ ) plus the five winners from the first round. Together with Australia, the OFC qualification contains $12$ teams. Fortunately, the design of the 2006 FIFA World Cup qualification (OFC) can be adopted without any changes:

First round

Format: single round-robin, 2 groups of five teams each, held in one country

Competitors: the $10$ lowest-ranked teams, that is, all nations except for Australia and New Zealand

Seeding: traditional⁶

Second round

Format: single round-robin, one group of six teams, held in one country

Competitors: the two highest-ranked teams (Australia, New Zealand) + the group winners and the runners-up in the first round

Seeding: redundant

Third round

Format: knockout

Competitors: the group winner and the runner-up in the second round

Seeding: redundant

The winner advanced to the inter-confederation play-offs.

The simulation has been carried out with 2.5 million independent runs in each scenario (e.g. when Australia plays in the OFC zone). The probability of qualification is provided by the mean of these iterations, for example, it equals 60% if a national team qualifies in 1.5 million runs. A further increase does not reduce statistical errors considerably and would be a futile exercise anyway in view of the warnings above.

A reasonable measure of unfairness

The degree of unfairness can be quantified by ranking the teams according to their Elo rating and summing the differences of qualifying probabilities that do not fall into line with this ranking. However, in order to ensure the comparability of different tournaments, it is worth guaranteeing independence of the number of competitors $n$ and the number of qualifying slots $q$ .⁷ Since the worst case with respect to fairness is when the $q$ lowest-ranked teams qualify with probability one, the crude sum is divided by its theoretical maximum of $(n - q) q$ to get a normalised measure between zero and one.

Definition 1

Measure of unfairness:

UF = \sum_{E_{i} \geq E_{j}} \frac{max {0; p (j) - p (i)}}{(n - q) q},

(7)

where

E_{i}

and

p (i)

are the Elo rating and the probability of qualification for team

i

, respectively,

n

is the number of competitors and

q

is the number of qualifying slots.

Formula (7) only considers the ordinal strength of the teams because prescribing how the differences in Elo rating should be converted into differences in qualifying probabilities would require further assumptions, which seem to be challenging to justify.

Results

The three main research questions, presented in the Introduction, will be discussed separately.

Quantifying unfairness

Figure 1 shows the probability of qualification for the 2018 FIFA World Cup as the function of Elo ratings. Unsurprisingly, the simple round-robin format of the CONMEBOL qualification guarantees that this tournament metric depends monotonically on the strength of the teams. The structure of the OFC qualification does not necessarily satisfy the fairness condition but it still holds because only the four weakest teams should play in the first round and the seeding is based on the strengths of the teams. Similarly, the AFC and CONCACAF qualifications are essentially conforming to the principle of giving higher chances for better teams, too.

Figure 1.

The probability of qualification for the 2018 FIFA World Cup.

As Table 1 reveals, the qualification tournaments of all confederations are constructed fairly (the slot in the play-offs is counted as half). The negligible numbers for AFC and CONCACAF are only due to the stochastic nature of the simulation, which leads to volatile qualifying probabilities for weak teams. A team with a lower Elo rating never has a higher chance to qualify than a team with a higher Elo rating if either of these probabilities is above 0.002%. On the other hand, some teams have the same Elo rating (see Tables A.1 and A.2 in the Appendix), and these cases naturally increase $UF$ according to Definition 1.

Table 1.

The level of unfairness within the confederations.

Confederation	Value of $UF$
AFC	$6.23 \times 10^{- 7}$
CONCACAF	$4.18 \times 10^{- 6}$
CONMEBOL	0
OFC	0

Unfairness has another dimension, that is, between the confederations. According to Figure 1, the South American qualifiers seem to be the toughest. In order to investigate this issue, Peru (the $7$ th strongest team in CONMEBOL, Elo: 1807) has been exchanged sequentially with the strongest teams in the other three confederations: Japan (AFC, Elo: 1757), Mexico (CONCACAF, Elo: 1893), and New Zealand (OFC, Elo: 1516). These countries are highlighted in Figure 1.

Table 2 reports the probabilities of qualification for the four nations if Peru would contest in various confederations. The numbers in the diagonal show that any team is the worst off when playing in the CONMEBOL qualifiers but the chance of Peru to participate in the 2018 FIFA World Cup would more than triple by playing in the AFC or CONCACAF zone. Being a member of the OFC would be less beneficial for Peru due to the lack of a direct qualification slot. Its effect can be seen to some extent in the qualifying probabilities of Japan and Mexico: since Peru would qualify with more than 97% probability from the OFC qualifiers to the inter-confederation play-offs, the two teams would have a larger probability to face Peru there, which would reduce their chances to advance to the World Cup finals.

Table 2.

Qualifying probabilities when Peru is moved to another confederation.

		Peru plays in
Team	Original confederation	AFC	CONCACAF	CONMEBOL	OFC
Japan	AFC	0.117	0.793	0.794	0.785
Mexico	CONCACAF	0.968	0.508	0.968	0.966
Peru	CONMEBOL	0.887	0.877	0.235	0.455
New Zealand	OFC	0.185	0.183	0.185	$5.67 \times 10^{- 5}$

A potential improvement of fairness

The straightforward solution to handle unfairness between the confederations would be to reallocate the slots available for them, especially because the current allocation system lacks any statistical validation, does not ensure the qualification of the best teams in the world, and does not reflect the number of teams per federation.³ The whole process is far from being transparent and is mainly determined by political, cultural, and historical factors. Consequently, operations research has a limited role to influence the allocation of World Cup slots between the FIFA confederations.

However, the matching in the two inter-confederation play-offs is probably a variable to be chosen freely by the FIFA executives who are responsible for the tournament design, as illustrated by the two policies used recently. We have considered three possibilities:

Random draw for the play-offs: the four participants from the confederations AFC, CONCACAF, CONMEBOL, and OFC are drawn randomly into two pairs;

Fair draw for the play-offs: the four participants are paired such that AFC vs. CONCACAF and CONMEBOL vs. OFC;

Close draw for the play-offs: the four participants are paired such that AFC vs. OFC and CONCACAF vs. CONMEBOL.

The random draw has been used in the 2006 FIFA World Cup qualification, as well as since the 2014 FIFA World Cup qualifiers. The close draw has been used in the 2010 FIFA World Cup qualification competition: it matches nations from closer time zones, which allows for better kick-off times. Therefore, it can be optimal for the players, as well as may maximise gate revenue and the value of television rights. Finally, the fair draw is inspired by Figure 1 and Table 2 since the CONMEBOL team is usually the strongest and the OFC team is usually the weakest in the play-offs.

The draws for the play-offs are compared in Table 3, the measure of unfairness (Definition 1) is presented in Table 3a, while Table 3b provides the probability of qualification for some countries. Again, there are some countries with the same Elo rating: Guatemala, Honduras (1539); Qatar, Trinidad and Tobago (1525); Jamaica, Oman (1522); Canada, El Salvador (1507); Haiti, Kuwait, North Korea (1446); Malaysia; Philippines (1225); India, Maldives (1068). These cases naturally increase the value of $UF$ , mostly when the national teams play in different confederations.

Table 3.

Unfairness and the draw for the play-offs

a. The overall level of unfairness
Draw policy for the play-offs
Random	Fair	Close
0.01063	0.00985	0.01308
b. The qualifying probabilities of certain teams
Team	Confederation	Elo rating	Draw policy for the play-offs
Team	Confederation	Elo rating	Random	Fair	Close
Australia	AFC	1727	0.708	0.744	0.781
Japan	AFC	1757	0.744	0.778	0.810
Mexico	Concacaf	1893	0.968	0.975	0.958
Peru	Conmebol	1807	0.235	0.241	0.205
New Zealand	OFC	1516	0.184	0.041	0.179

Intuitively, the fair draw is the least unfair. The close draw mostly favours AFC and OFC, however, it is detrimental to the members of the CONCACAF and CONMEBOL, implying the most severe unfairness.

The effect of a fair draw is detailed in Figure 2 for the teams with at least one percentage point change in the probability of qualification to the 2018 FIFA World Cup. Compared to the current random design, all Asian and South American countries would be better off and the strongest CONCACAF countries are also preferred. On the other hand, all nations of the OFC, in particular, the dominating New Zealand, would lose substantially from this reform. The gains are distributed more equally because there is no such a prominent team in the other zones. Some weaker CONCACAF members are worse off due to the impossibility of playing against New Zealand in the inter-confederation play-offs.

Figure 2.

The effect of improving the fairness of the 2018 FIFA World Cup qualification via modifying the draw for inter-confederation play-offs.

Counterfactual: was it favourable for Australia to join the AFC?

FIFA president Sepp Blatter had promised a full slot to the OFC as part of his re-election campaign in November 2002 but the suggestion was reconsidered in June 2003⁶⁰. Subsequently, the largest and most successful nation of the OFC, Australia, left to join the AFC in 2006. It raises the interesting question of how this move has affected the 2018 FIFA World Cup qualification.

First, the unfairness measure $UF$ would be $0.013$ with Australia playing in the OFC, which corresponds to an increase of more than 20% compared to the current situation. The action of Australia has contributed to the fairness of the 2018 FIFA World Cup qualification. Furthermore, the magnitude of the improvement is higher than the gain from the proposed novel draw for the inter-confederation play-offs. This reduction in unfairness likely explains why the move of Australia was approved.

Second, the probabilities of qualification are computed if Australia would have remained in the OFC. Figure 3 plots the effects for the national teams facing a change of at least one percentage point. Notably, Australia has increased the probability of participating in the 2018 FIFA World Cup from 43% to 70% by leaving the OFC for the AFC. The move has also been strongly favourable for New Zealand, which is now the strongest OFC team and has more than 70% chance to grab the slot guaranteed in the play-offs for Oceania. Every CONCACAF and CONMEBOL member has been better off due to the reduction in the expected strength of the countries contesting in the play-offs. However, all original AFC nations have lost with the entrance of Australia, especially those teams that are only marginally weaker than Australia.

Figure 3.

The effect of Australia being a member of the AFC on the 2018 FIFA World Cup qualification.

Conclusions

We have analysed four series of qualification tournaments for the 2018 FIFA World Cup via Monte Carlo simulations. Their design does not suffer from serious problems. However, there are substantial differences between the chances of nations playing in different continents: Peru would have tripled its probability of qualification by competing in the AFC or CONCACAF zone, whereas New Zealand would have lost any prospect of participation by being a member of the CONMEBOL. Australia is found to have greatly benefited from leaving the OFC for the AFC in 2006.

In addition, probably first in the literature, a measure of unfairness has been proposed to quantify to which extent weaker teams are preferred over stronger teams by a tournament design. A simple modification in the format of the inter-confederation play-offs can reduce this metric by about 7.5%, hence FIFA has a good reason to consider the novel policy.

Hopefully, this paper will become only a pioneering attempt of academic researchers to study the qualification to the soccer World Cup. There remains huge scope for improving the simulation model, especially concerning the prediction of individual matches. Nonetheless, even these results might be useful for sports governing bodies: we believe that FIFA could further increase the economic success of World Cups by using a more transparent and statistically validated method in the allocation of qualifying berths and the design of confederation-level qualification tournaments.

Footnotes

Acknowledgements

This paper could not have been written without my father (also called László Csató), who has primarily coded the simulations in Python.

We are grateful to Mark Gagolewski, Dries Goossens, Lars Hvattum, and Cristophe Ley for useful advice.

Five anonymous reviewers provided valuable comments and suggestions on earlier drafts.

We are indebted to the Wikipedia community for collecting and structuring valuable information on the sports tournaments discussed.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The research was supported by the MTA Premium Postdoctoral Research Program grant PPD2019-9/2019.

ORCID iD

László Csató

Notes

Appendix

Table A.4.

OFC nations participating in the 2018 FIFA World Cup qualification

Country	Elo rating of 30 August 2015
New Zealand	1516
New Caledonia	1357
Tahiti	1226
Solomon Islands	1175
Vanuatu	1174
Fiji	1281
Papua New Guinea	1053
Samoa	681
Tonga	684
American Samoa	381
Cook Islands	675

The countries are ranked according to the FIFA World Ranking of July 2015, separately for the positions $1$ – $7$ (byes in the first round) and $8$ – $11$ (playing in the first round). The first match in the first round was played on 31 August 2015.

The team written in italics qualified for the inter-confederation play-offs of the 2018 FIFA World Cup.

References

Palacios-Huerta

. Beautiful Game Theory: How Soccer Can Help Economics. Princeton: Princeton University Press, 2014.

Frawley

Van den Hoven

. Football Participation Legacy and Australia’s Qualification for the 2006 Football World Cup. Soccer & Society 2015; 16(4): 482–492.

Stone

Rod

. Unfair Play in World Cup Qualification? An Analysis of the 1998–2010 FIFA World Cup Performances and the Bias in the Allocation of Tournament Berths. Soccer & Society 2016; 17(1): 40–57.

FIFA. FIFA Council confirms contributions for FIFA World Cup participants. http://web.archive.org/web/20200726200924/https://www.fifa.com/who-we-are/news/fifa-council-confirms-contributions-for-fifa-world-cup-participants-2917806 (2017, accessed 27 October).

Depetris-Chauvin

Durante

Campante

. Building Nations Through Shared Experiences: Evidence From African Football. American Economic Review 2020; 110(5): 1572–1602.

Flégl

. Performance Analysis During the 2014 FIFA World Cup Qualification. Open Sports Sci J 2014; 7: 183–197.

Durán

Guajardo

Sauré

. Scheduling the South American Qualifiers to the 2018 FIFA World Cup by Integer Programming. Eur J Oper Res 2017; 262(3): 1109–1115.

Pollard

Armatas

. Factors Affecting Home Advantage in Football World Cup Qualification. International Journal of Performance Analysis in Sport 2017; 17(1-2): 121–135.

Csató

. The Incentive (in)compatibility of Group-based Qualification Systems. Int J Gen Syst 2020; 49(4): 374–399.

10.

Csató

. When Neither Team Wants to Win: A Flaw of Recent UEFA Qualification Rules. Int J Sports Sci Coach 2020; 15(4): 526–532.

11.

Dagaev

Sonin

. Winning by Losing: Incentive Incompatibility in Multiple Qualifiers. J Sports Econom 2018; 19(8): 1122–1146.

12.

Vong

AIK

. Strategic Manipulation in Tournament Games. Games Econ Behav 2017; 102: 562–567.

13.

Csató

. Tournament Design: How Operations Research Can Improve Sports Rules. Palgrave Pivots in Sports Economics Cham, Switzerland: Palgrave Macmillan, 2021.

14.

Groh

Moldovanu

Sela

Sunde

. Optimal Seedings in Elimination Tournaments. Economic Theory 2012; 49(1): 59–80.

15.

Arlegi

Dimitrov

. Fair Elimination-type Competitions. Eur J Oper Res 2020; 287(2): 528–535.

16.

Haugen

Krumer

. On importance of tournament design in sports management: Evidence from the UEFA Euro 2020 qualification. In: Ratten V (ed) Innovation and Entrepreneurship in Sport Management. New York: Edward Elgar Publishing, 2021, pp.22–35.

17.

Krumer

Megidish

Sela

. First-mover Advantage in Round-robin Tournaments. Soc Choice Welfare 2017; 48(3): 633–658.

18.

Krumer

Megidish

Sela

. The Optimal Design of Round-robin Tournaments with Three Players. J Sched 2020; 23(3): 379–396.

19.

Laica

Lauber

Sahm

. Sequential Round-robin Tournaments with Multiple Prizes. Games Econ Behav 2021; 129: 421–448.

20.

Sahm

. Are Sequential Round-robin Tournaments Discriminatory?. Journal of Public Economic Theory 2019; 21(1): 44–61.

21.

Krumer

Lechner

. First in First Win: Evidence on Schedule Effects in Round-robin Tournaments in Mega-events. Eur Econ Rev 2017; 100: 412–427.

22.

Guyon

. Risk of Collusion: Will Groups of 3 Ruin the FIFA World Cup?. Journal of Sports Analytics 2020; 6(4): 259–279.

23.

Apesteguia

Palacios-Huerta

. Psychological Pressure in Competitive Environments: Evidence From a Randomized Natural Experiment. American Economic Review 2010; 100(5): 2548–2564.

24.

Vandebroek

McCann

Vroom

. Modeling the Effects of Psychological Pressure on First-mover Advantage in Competitive Interactions: The Case of Penalty Shoot-outs. J Sports Econom 2018; 19(5): 725–754.

25.

Anbarcı

Sun

C-J

Ünver

. Designing Practical and Fair Sequential Team Contests: The Case of Penalty Shootouts. Games Econ Behav 2021; 130(0): 25–43.

26.

Brams

Ismail

. Making the Rules of Sports Fairer. SIAM Rev 2018; 60(1): 181–202.

27.

Csató

. A Comparison of Penalty Shootout Designs in Soccer. 4OR 2021; 19(5): 183–198.

28.

Csató

Petróczy

. Fairness in penalty shootouts: Is it worth using dynamic sequences? Manuscript; 2022. arXiv: https://arxiv.org/abs/2004.092252004.09225.

29.

Lambers

Spieksma

FCR

. A Mathematical Analysis of Fairness in Shootouts. IMA J Manag Math 2021; 32(4): 411–424.

30.

Palacios-Huerta

. Tournaments, Fairness and the Prouhet-Thue-Morse Sequence. Econ Inq 2012; 50(3): 848–849.

31.

Guyon

. What a Fairer 24 Team UEFA Euro Could Look Like. Journal of Sports Analytics 2018; 4(4): 297–317.

32.

Vaziri

Dabadghao

Yih

Morin

. Properties of Sports Ranking Methods. J Oper Res Soc 2018; 69(5): 776–787.

33.

Dyte

Clarke

. A Ratings Based Poisson Model for World Cup Soccer Simulation. J Oper Res Soc 2000; 51(8): 993–998.

34.

Deutsch

. Looking Back At South Africa: Analyzing and Reviewing the 2010 FIFA World Cup. Chance 2011; 24(2): 15–23.

35.

Groll

Schauberger

Tutz

. Prediction of Major International Soccer Tournaments Based on Team-specific Regularized Poisson Regression: An Application to the FIFA World Cup 2014. J Quant Anal Sports 2015; 11(2): 97–115.

36.

O’Leary

. Crowd Performance in Prediction of the World Cup 2014. Eur J Oper Res 2017; 260(2): 715–724.

37.

Groll

Ley

Schauberger

Van Eetvelde

. A Hybrid Random Forest to Predict Soccer Matches in International Tournaments. J Quant Anal Sports 2019; 15(4): 271–287.

38.

Groll

Ley

Schauberger

Van Eetvelde

Zeileis

. Hybrid machine learning forecasts for the FIFA Women’s World Cup 2019. Manuscript, 2019; arXiv: https://arxiv.org/abs/1906.011311906.01131.

39.

Groll

Hvattum

Ley

Popp

Schauberger

Van Eetvelde

Zeileis

. Hybrid machine learning forecasts for the UEFA EURO 2020. Manuscript. 2021; arXiv: https://arxiv.org/abs/2106.057992106.05799.

40.

Jones

. The World Cup Draw’s Flaws. The Mathematical Gazette 1990; 74(470): 335–338.

41.

Rathgeber

. Why Germany was Supposed to Be Drawn in the Group of Death and why it Escaped. Chance 2007; 20(2): 22–24.

42.

Scarf

Yusof

. A Numerical Study of Tournament Structure and Seeding Policy for the Soccer World Cup Finals. Stat Neerl 2011; 65(1): 43–57.

43.

Guyon

. Rethinking the FIFA World Cup^TM Final Draw. J Quant Anal Sports 2015; 11(3): 169–182.

44.

Laliena

López

. Fair Draws for Group Rounds in Sport Tournaments. Int Trans Oper Res 2019; 26(2): 439–457.

45.

Cea

Durán

Guajardo

Sauré

Siebert

Zamorano

. An Analytics Approach to the FIFA Ranking Procedure and the World Cup Final Draw. Ann Oper Res 2020; 286(1-2): 119–146.

46.

Scarf

Yusof

Bilbao

. A Numerical Study of Designs for Sporting Contests. Eur J Oper Res 2009; 198(1): 190–198.

47.

Lasek

Szlávik

Bhulai

. The Predictive Power of Ranking Systems in Association Football. International Journal of Applied Pattern Recognition 2013; 1(1): 27–46.

48.

Baker

McHale

. Time-varying Ratings for International Football Teams. Eur J Oper Res 2018; 267(2): 659–666.

49.

Van Eetvelde

Ley

. Ranking methods in soccer. In Kenett RS, Longford TN, Piegorsch W, and Ruggeri F, (eds) Wiley StatsRef: Statistics Reference Online, Hoboken, New Jersey, USA: Springer, 2019, pp.1–9.

50.

Ley

Van de Wiele

Van Eetvelde

. Ranking Soccer Teams on the Basis of Their Current Strength: A Comparison of Maximum Likelihood Approaches. Stat Modelling 2019; 19(1): 55–73.

51.

Corona

Forrest

Tena

Wiper

. Bayesian Forecasting of UEFA Champions League Under Alternative Seeding Regimes. Int J Forecast 2019; 35(2): 722–732.

52.

CONCACAF. Concacaf Announces Format for the 2022 FIFA World Cup Confederation Qualifiers. 10 July 2019). https://www.concacaf.com/en/world-cup-qualifying-men/article/concacaf-announces-format-for-the-2022-fifa-world-cup-confederation-qualifiers.

53.

Hvattum

Arntzen

. Using ELO Ratings for Match Result Prediction in Association Football. Int J Forecast 2010; 26(3): 460–470.

54.

Lasek

Szlávik

Gagolewski

Bhulai

. How to Improve a Team’s Position in the FIFA Ranking? A Simulation Study. J Appl Stat 2016; 43(7): 1349–1368.

55.

FIFA. 2026 FIFA World Cup^TM: FIFA Council designates bids for final voting by the FIFA Congress. http://web.archive.org/web/20210306161039/https://www.fifa.com/who-we-are/news/2026-fifa-world-cuptm-fifa-council-designates-bids-for-final-voting-by-the-fifa- (2018, 10 June).

56.

FIFA. Revision of the FIFA / Coca-Cola World Ranking. 2018. https://img.fifa.com/image/upload/edbm045h0udbwkqew35a.pdf.

57.

Chater

Arrondel

Gayant

J-P

Laslier

J-F

. Fixing Match-fixing: Optimal Schedules to Promote Competitiveness. Eur J Oper Res 2021; 294(2): 673–683.

58.

Dagaev

Rudyak

. Seeding the UEFA Champions League Participants: Evaluation of the Reform. J Quant Anal Sports 2019; 15(2): 129–140.

59.

Football rankings. Simulation of scheduled football matches. http://www.football-rankings.info/2020/12/simulation-of-scheduled-football-matches.html (2020, 28 December).

60.

ABC. Farina blasts FIFA World Cup backflip. Australian Broadcasting Corporation (ABC) News. 2003; 29 June. https://www.abc.net.au/news/2003-06-29/farina-blasts-fifa-world-cup-backflip/1878086.