Factors associated with match outcomes in elite European football – insights from machine learning models

2.1 Data extraction

We extracted the fixtures data of teams competing in the top seven European leagues, i.e., the English Premier League (EPL), the French Ligue 1, the German Bundesliga, the Dutch Eredivisie, Italian Serie A, the Portuguese Liga and Spanish Liga. This includes 21 seasons from 2001/02 to 2021/22. The fixtures from all in-season competitions were considered, i.e., leagues and domestic, European and international cups. This data was sourced from Transfermarkt. Across the 21 seasons, this represents 43 competitions, 269 teams and more than 61,000 fixtures.

2.2 Model-based analysis

Based on the above-mentioned fixtures data extract, the following metrics were used to identify the key factors that could impact teams performance:

Rest days: number of days between matches, irrespective of the competition. For instance, playing on Wednesday after a previous match on Sunday gives two rest days (Monday and Tuesday). When data was erroneously entered in the online database (e.g., 2 games played the same day), data was changed to NaN and treated as a missing value (0.01% of the dataset).

Sixteen groups of data were used for the final analysis (all regrouping data from the 2000/01 to the 2021/22 seasons). The 16 different models run were the following:

Main competitions all together: domestic leagues, UEFA Champions League (CL) and Europa League (EL) including their qualifying phases.

For the total 20 year period considered there were approximately 53,000 fixtures, with about 24,000 fixtures in the first decade (2001/02 to 2009/10) and 29,000 in the second (2010/11 to 2021/22). Domestic league fixtures accounted for 47,000 fixtures, while there were ∼6,100 fixtures from the Champions League and Europa League competitions. Using the ELO rating we defined the top 10% teams as those having an ELO rating greater than 1624, the 90th percentile of ELO rating over the whole period. When limiting to top teams playing against each other we had about 880 fixtures. The level of missing data was not about 5% for most metrics and always <12%. Missing data was mostly observed in international cup fixtures, and almost only for the teams not playing in the top 7 leagues (weaker teams). We did not remove fixtures with missing data and let the model handle those cases as the level was the same for the training and test sets.

In order to study the relationship between result (win, draw, loss) and the explanatory variables, we used eXtreme Gradient Boosting (XGBoost) for Classification. We focused on the Win class to analyse the factors related to performance.

Even if multicollinearity has no impact on performance of XGBoost models by nature, it does affect feature importance. Thereby, we studied the Pearson correlation between all pairs of features. The largest positive correlation value was between Difference Elo and Difference Domestic Performance to Date (0.76), while the largest negative one was between Home and Travel Distance (−0.77). Considering that there was no very strong (positive or negative) correlation between any pairs of features, we decided to keep all of them in the model.

When looking at the overall context, the level of missing data per metric goes from 0 to 10%. Information about competitions and Elo is almost fully complete while the other metrics related to a team has 1 to 5% of missing data. For metrics related to the difference with the opposition team, this doubles i.e. up to 10%. This lack of data mainly comes from teams that are not part of the top 7 European leagues, supposedly weaker. This phenomenon then impacts less league-focused models i.e. less than 5% of missing data for each metric, while being more present for models based on European matches (30 to 60% of missing data). Considering that the data is missing not at random and this pattern is the same in the training and test sets, its impact on the model accuracy is not significant (Rusdah & Murfi, 2020). We then decided to run all models without imputation on them.

As teams played in empty stadiums from COVID restart in May 2020 to the end of the 2020/21 season—impacting the home-field advantage—we decided to exclude this period from the analysis. For the overall model (main competitions), we then split the data set into training and test sets based on the following seasons:

Training set (80% of the data): from 2001/02 to 2016/17

We built the training and test sets of the other contexts following this 80% /20% rule.

The XGBoost model structure is defined by hyper-parameters. In order to get the optimal ones, we used a randomised grid-search based on the training set. A grid-search runs through all the different parameters fed into the parameter grid and produces their best combination based on a scoring metric. A randomised grid-search does not try all combinations out but rather a fixed number of iterations, i.e., 20 here. It then returns a relatively accurate performing model in a significantly shorter runtime (Bergstra, 2012). The parameter grid considered is the following one:

Number of iterations: 20

The values with an asterisk are the optimal ones for the overall model.

To explain how our model works, i.e. the decisions it is making, we used SHapley Additive exPlanations (SHAP) values (Lundberg & Lee, 2017). SHAP does this by using fair allocation results from cooperative game theory to allocate credit for a model’s output among its input features. Its calculation involves averaging the marginal contributions of each player (or feature) across all potential permutations of players (features). This involves assessing every possible combination of features and determining the impact each feature has on the model’s prediction when included in these combinations. By averaging these contributions across all possible feature arrangements, it achieves a balanced and interpretable evaluation of each feature’s importance in the model’s prediction. One of the fundamental properties of SHAP values is that all the input features will always sum up to the difference between baseline (expected) model output and the current model output for the prediction being explained. In the case of binary classification the sum of the feature SHAP values equals the prediction log-odds, while for multiclass classifications the softmax function is used. As such the individual SHAP values are difficult to interpret by themselves. As our task is defined from a team perspective, we implemented a 3-class classification: win, draw or loss. We then decided to study the following concepts related to the “win” class i.e. the factors that contribute to teams’ success:

Feature Importance: based on the absolute SHAP values per feature, it analyses the global importance of each feature. For a given prediction each feature will have a positive or negative SHAP value indicating whether it positively or negatively contributes towards the predicted value. The absolute SHAP value is a measure of how important the feature is irrespective of its direction, and so taking its average across all predictions provides a global measure of how important a feature is to the model.

Based on this Feature Dependence concept, we created a modified version of the SHAP dependence plot (Lundberg, 2018) (i.e., all Figures showing univariate results). One dot is the SHAP value (y-axis) related to an actual observation of a given metric (x-axis) –e.g. travel distance –within a given context i.e. match here. For instance, for many matches, travel distance is equal to 500 kms multiple times. This does not mean that all of these points get the same SHAP value as they come from different matches. For each game, travel distance = 500 kms can have a different impact on the expected result depending on the other metrics’ value. When the feature (x-axis) is non-binary, local trends between two non-binary entities are shown using locally-linear estimated scatterplot smoothing (LOESS) and its 95% prediction intervals (PI). We defined four zones to summarise the impact of the feature on match-to-match rotations:

Increasing: both the LOESS curve and PI are above 0

For binary features, a box plot is used. It is based on the SHAP values related to each instance (0 and 1), and the same parameters as the ones defined in the above-mentioned Descriptive Analysis subsection. The four zones are created as follows:

Increasing: both the median and interval between whiskers (WI) are above 0

Finally, we evaluated the probabilistic prediction of an individual match using the ranked probability score (RPS). In order to assess the model performance on the entire test set, we defined the average over all RPS related to each match in this set. The smaller the average RPS, the better the predictions (Berrar, Lopes & Dubitzky, 2019). We compared our model trained on all listed features with:

Naive model: all predictions are the relative frequencies of wins, draws and losses in the training set.

Knowing that the model performance can vary depending on the test set, we generated 95% confidence intervals (CI) for the average RPS by bootstrapping the test set. As seen on Fig. 1, the full model related to the overall context (main competitions) performs better than both the naive and Elo models, with an average evaluation of 0.1990, 95% CI being [0.1971, 0.2009]. Even if our training and test sets are different from the ones of the 2017 Soccer Prediction Challenge, this model performance is better than the one from the challenge winners Team OH whose average RPS was 0.2063.

OPEN IN VIEWER

Fig. 1

Average Ranked Probability Score of the model trained on all metrics, the one trained only on Elo variables and the naive one.

We use SHAP to interpret the model predictions. Its additive nature allows each prediction to be split out into the contributions from each feature in the model over and above the SHAP expected value, and ranked in order of importance / magnitude.

The SHAP expected value is essentially the frequency of a team winning the game across the training set. Considering our multi-class classification, the softmax function should be applied to this number in order to transform it into an understandable one. The softmax function turns a vector of K (K = 3 here) real values into a vector of K real values that sum to 1. The expected value of winning is 0.597 here, which translates to a frequency of winning the game for each team (without taking the home-field advantage into account) of 36.6%. Figure 2 shows how SHAP is used to interpret the model prediction of Liverpool winning their game at home against Brentford on January 16, 2022. The combined SHAP contributions plus the expected value gives the resulting prediction, 1.473 in this case which gives a 69.3% chance of winning the game. The main contributor here was the Elo difference between Liverpool and their opponents i.e. 318 before the match. It added a contribution of 0.33 in addition to the expected value. The contribution of the other features is shown in Fig. 5. Liverpool actually won that game 3-0.

OPEN IN VIEWER

Fig. 2

SHAP contribution of each feature on Liverpool performance for their game at home against Brentford on January 16, 2022.

4.1. Match location and travel distance

The average contribution to the 10% best teams model was slightly higher for travel distance (0.085) than location (0.078) (Fig. 5); their contribution magnitude was however twice that of the second sets of metrics (i.e; Elo, see below). The SHAP dependency plot in Fig. 7 shows that while match outcomes were clearly decreased away (right panel), it’s essentially the away matches that require to travel over distances >100 km that was really likely to affect performance (left panel) equivalent to a drive of 1 h–1 h30 maximum in bus. The substantial and important effect of match location is in agreement with the commonly reported home-field advantage (Pollard, 1986; Lago-Peñas, 2016). While there remains some inconsistency in the literature, we recently reported a clearly lower likelihood to win for away vs. home (expected 30% vs. 45% wins, Buchheit, 2022 & 2023b) within the same present data set. Among all the possible factors explaining the home-advantage, travel fatigue and crowd support were shown to contribute less to home advantage than do the less easily quantifiable benefits of familiarity with conditions when playing at home (Lago-Peñas, 2016).

4.2 Elo

The contribution of Elo to team performance was straightforward (with an overall contribution to the 10% top teams model of 0.045, Fig. 5), with top teams (Elo > 1624, right panel Fig. 8) performing better than others. Performance was also increased as soon as when the absolute difference in Elo vs. the opponent was positive (Fig. 8, left panel) (contribution of 0.035, Fig. 5). These findings are logical are confirm the relevance of this index as a measure of overall team quality. The direct application of these findings is that coaches may expect ‘easier’ wins when playing teams of clearly lower Elo, and may use this as an opportunity to rotate/rest players when needed (i.e., congested periods).

4.3 Difference in recent performance and domestic performance to date

The overall contribution of team performance (recent and to date) to the 10% best teams model ranged from 0.015 to 0.030 (Fig. 5). We observed various associations between actual match outcomes and differences in both domestic performance to date (bottom left, Fig. 5) and differences in recent performance (bottom right, Fig. 5). In practice, top teams that have been performing better (>0% of possible points) than their opponents both in their domestic league since the start of the season (bottom left) and over the last five matches (bottom right) showed improved performance. While overall team performance (since 1980 for our algorithm, Settembre, 2023) is used to produce the Elo rating (discussed above), the inclusion of these short- (five last matches) and long-term (performance to date since the start of the season) metrics was aimed at capturing the actual ‘form’ of each team which carries some sort of performance momentum. Anecdotally, this recent performance over the most 5 recent matches (Radzimiński, 2022) is also used in many sport websites for the general population of readers (i.e., google sport results). It is worth noting that we used a generic definition of ‘recent performance’, which can be interpreted as good or bad for a particular team depending on their context at the time. For example, a team in the relegation zone would be described as being in ‘good form’ if they picked up 10 points from 5 games, but for a title-chasing team the same outcome would probably be considered ‘bad form’. We chose this approach with practicality in mind since the quality of performance is team-dependent and may also change with time (and seasons). Defining ‘recent performance’ at the team level is a topic for future investigation.

4.4 Managers

The contribution of managers lifespan to the 10% best team model was 0.017 (Fig. 5), i.e., half of that of Elo and a fourth of travel distance and match location. Interestingly, the number of matches since the manager was in charge was ranked as a substantial contributor to performance in many models. It was in the Top 3 in the teams in European cups (Table 1), and in 7th position in the top 10% teams model (Figs. 5 and 6). In this latter model, teams had a poorer performance when playing with a recently-appointed (<30 matches, which are likely played over a little bit less than one season approximately) or long-established (>420 matches, which can represent >7 years) manager. The 30 matches mark is consistent with the idea that coaches need around a season to build their team tactics and game model, and confirms the results of a recent study when the influence of coach replacement was examined (Radzimiński, 2022). They actually reported that the highest number of collected points per game are obtained by coaches who lead their teams for several seasons. In contrast, these results also showed that while changing the coach during the soccer season may result in short-term improvement in team results and physical match performance, after a period of approximately 5 games, this effect disappears (Radzimiński, 2022).

While it’s also intuitive that coaches’ influence may decline over prolonged time (e.g., lack of new insights, decreased player motivation and engagement in relation to management style), the 8-year mark on the right side of the graph (Fig. 10) may be at first perceived at odd with current practices, where the majority of coaches last only a few seasons (e.g., the average life-span of managers ranges between 772 days, which is a little over two years and one month in the EPL, 628 days in the Bundesliga, 617 days in La Liga, and just 385 and 384 days in Ligue 1 and Serie A, respectively) (Price, 2022). A closer look at the graph (fewer data points) suggest that this trend is essentially reflective of the rare exceptions of managers who reached that milestone; the lower precision of the model at this point (as shown by the large CI around the main regression line) may be considered when interpreting this trend.

4.5 Rest days

While only ranked as the 10th and 8th contributing factor when looking at all competitions together (Fig. 3) or only at the top 10 teams (with contribution to the model of 0.05 to 0.015, Fig. 5), respectively, the contribution of rest days (Fig. 11) was straightforward and consistent with previous studies (Buchheit, 2022; Verheijen, 2012). The data confirm that performance is clearly impaired with only two days of rest (e.g., playing on Wednesday and then again on Saturday, Fig. 12, left panel) even in top teams that may have more options to rotate players (Buchheit, 2022). Further than the potential physical and mental fatigue that may not be completely recovered within 48 h (Nedelec, 2012), we also observed that when playing the next match within less than three rest days, match-to-match rotations were increased (Settembre, 2023). In this complex context, coaches may rotate more players than usual to avoid increasing their risk of injuries (Page, 2023), which as discussed above, decreases between-player communication, and in turn, match outcomes.

It’s only with ≥4 days of recovery that top teams seem to regain their expected competitive advantage (in relation to their status and Elo, see above) (Buchheit, 2022). Along the same lines, it’s when these top 10% of teams had ≥1 rest day less than their opponents that they tend to underperform (Fig. 11, right panel).

4.6 Line-up stability and match-to-match rotations

The contribution of line-up stability and match-to-match rotations to the 10% best teams model was around 0.010 for each metric (Fig. 5), i.e., a third of that of Elo and a eighth of travel distance and match location. Teams performed better when they presented fewer rotations (i.e., <0) from the previous match than their opponent (Fig. 12 left panel). The detrimental effect of player’s rotations is straightforward, considering that line-up stability favours better between-player communication (Verheijen, 2022) and in turn, match outcomes. In a previous study examining the effect of rotation on the performance of Champions’ League teams playing three matches a week (2014–15 to 2017–18 competitive season), more points in the domestic league were lost when a large number of players participated in the initial list for the three games. Similarly, increasing rotations in the initial list between the 1st and the 3rd game and between the 2nd and the 3rd game had a negative effect on the domestic league performance (Bekris, 2020). A specific analysis of 98 different teams in the “Top 5” European domestic leagues over the 2016–17 season showed more contrasted results however (i.e., lack of clear correlations between rotation numbers and performance); this lead the author to suggest that the effect of rotations per se is more complex than and that and likely related to a myriad of other factors (e.g., opposition, travels –which gives more credit to the present multi-factorial analysis), and should be relied upon on a case-by-case basis (Schmidt, 2017). For instance, in another study (Settembre, 2023), we showed that rotations were slightly increased when playing away, which may be related to the change in tactics that often occur away, with teams shifting toward more defensive team formations (Buchheit, 2023a). In accordance with the typical home-field advantage (e.g., crowd support, familiar environment, no travel, etc.) (Pollard, 1986; Lago-Peñas, 2016), this increased number of rotations when playing away may also partially explain the lower likelihood to win (expected 30% vs. 45% wins, for away vs. home, Buchheit, 2022). The current understanding about rotations, is that there is in fact a tradeoff between line-up stability and load management (i.e., to preserve players’ health, especially when playing congested); deciding how many players to rotate from one match to the next, and who to rotate remains a key question for every manager (O’Hanlon, 2020).

4.7 Limitations

The main limitation of the current analysis remains the data set obtained from an online database. This data set constrained the factors included in the modelling part, since there may be other factors that were not available from Transfermarkt, whose influence could therefore not be examined. We have not compared our results with other machine learning techniques that are applicable to the problem (ordinal logistic regression, k-nnand etc.). While performance improvements are possible with those approaches we are satisfied with the approach given its performance relative to other studies. Finally, our study involved an analysis of various contexts to assess the potential variation of factors across different scenarios. These contexts were deliberately designed for comparative purposes, spanning decades (comparing the first with the second), groups of competitions (domestic leagues versus European competitions), single competitions (contrasting one league with another, as well as the UEFA Champions League with the UEFA Europa League), and specific team levels versus the entirety of available data. Importantly, our methodology ensured no cherry-picking of data. In this paper, we present a comprehensive summary of all 16 contexts, a broader perspective drawn from the complete dataset, and a detailed exploration of the specific context surrounding the top 10% of teams. This approach aims to provide a thorough understanding of how factors may evolve or differ in diverse contexts.”