Barriers to coach decision-making during Australian football matches and how it can be supported by artificial intelligence

Abstract

Elite sport coaches are often expected to make high-stakes decisions in time-constrained environments, such as during matches. The quality or efficiency of these decisions is bound by context-specific constraints on their decision-making. Artificial intelligence-based tools could alleviate some of these constraints. In Australian football, it is not yet known what these context-specific constraints are, nor is there any understanding of the thoughts and perception of elite Australian football coaches towards the use of artificial intelligence to support their decision-making. This study is the first to explore these issues in the context of elite Australian football and aims to inform the effective design and integration of tools to support the in-match decision-making of coaches in a manner that is welcomed by coaches. Using semi-structured interviews with elite Australian football coaches (n = 6) and reflexive thematic analysis, two domains were explored: ‘barriers to decision-making’ and ‘thoughts and perceptions of a decision support system (DSS)’. Coaches discussed seven ‘cognitive’ and ‘environmental’ barriers, some of which may be addressed using artificial intelligence. Meanwhile, four themes were generated for the second domain, namely ‘willingness to adopt’, ‘expectation’, ‘concern and criticism’, and ‘DSS role’. The findings enable practitioners to design tools that are intended to address some of these barriers, while also providing insights into important considerations for the implementation and design of these tools that may increase the likelihood of achieving the endorsement of elite coaches and their support staff.

Keywords

Cognitive performance analysis team sport technology

Introduction

Elite sport coaches are responsible for making complex decisions regarding team selection, training plan, game plan, and athlete management.¹ In doing so, they strive to think rationally, considering the pros and cons of any decision and its consequences prior to acting.² According to bounded rationality, the outcome of a decision made by the coach is bound by the cognitive and environmental constraints on their rational thinking.³ Consequentially, rational thinking may lead coaches to decisions that are suboptimal, due to the incapacity for humans to perfectly analyse and rationalise every piece of information relevant to the decision.^4,5 This is particularly problematic for decisions made in time-constrained environments, such as during competition. However, there are opportunities to support the decision-making (DM) of coaches by minimising their DM constraints.^2,6

To minimise their DM constraints, one solution could be to develop and implement a decision support system (DSS). A DSS is an interactive computer-based system, often employing artificial intelligence (AI). It is designed to support the DM of users without them requiring an understanding of the complex algorithms underpinning the system.⁷ Typically, the goal is for the DSS to perform tasks and computations that can be done quickly and accurately by computers, in real-time, leading to information that improves the quality of decisions by the decision maker. Such systems have been proposed to provide pre- and post-competition support for coaches in decisions regarding athlete monitoring and performance evaluation, though rarely for in-match support.^8,9

Real-time performance analysis can play a critical role in supporting the in-match DM of coaches and could present an opportunity to be enhanced with a DSS.¹⁰ Watson et al.¹¹ developed a real-time DSS to provide tactical support to rugby union coaches during matches; however, they acknowledged at least two main challenges: firstly, the difficulty in implementing their system during matches due to concerns over the complexity and availability of the required volume of data; and secondly, persuading coaches to soften some of their firmly held beliefs about DM, to achieve their ‘buy-in’ or endorsement. The concept of ‘buy-in’ represents an important consideration for coaches looking to use new technologies as they assess whether it is helpful, trustworthy, and feasible to implement.¹² The technology acceptance model posits that acceptance of new technologies depends on two main factors: their perceived usefulness and their perceived ease of use.¹³ Thus, any DSS proposed for coaches in elite sport should be designed with these considerations in mind to increase the likelihood of achieving coach ‘buy-in’. While coach attitudes to new technologies have been explored more broadly, no research has explored the thoughts and perceptions of coaches towards an in-match DSS.

Australian football is a sport that embraces in-match performance analysis to support the DM of coaches.² With repeated in-match DM opportunities, access to live data feeds, and opportunities to provide feedback during the three scheduled breaks in play, as well as via a ‘runner’ throughout the match, Australian football presents an opportunity for the implementation of a real-time DSS. Aarons et al.² explored the DM process of elite Australian football coaches during matches, enabling researchers and practitioners to strategically design systems or interventions that facilitate improvements to one or more of the DM stages. However, this alone does not provide sufficient understanding of what issues coaches perceive to require decision support and how decision support would be received. Returning to the technology acceptance model, to achieve the ‘buy-in’ of coaches, it is necessary to address a perceived problem or barrier to DM, which would subsequently contribute to improvements in the perceived usefulness of the intervention. Additionally, to improve ease of use, it is necessary to understand the visual and functional design elements that would enable coaches to effectively utilise and interpret the information provided by a DSS. Thus, the following investigation aims to answer three research questions to improve the current understanding of the preferences of coaches and opportunities for improvement and intervention in relation to in-match DM support:

What barriers to effective DM do coaches face during matches?

What are the thoughts and perceptions of coaches towards the use of AI-based DSSs during matches?

What are the visual and functional design elements that would enable coaches to utilise and interpret information provided by a DSS?

Materials and methods

Participants and data collection

The present study was part of a larger qualitative study investigating the DM of elite Australian football coaches during matches.² Semi-structured interviews were conducted with six elite Australian football head coaches (all employed at the time of data collection), with at least 1 year of experience in their role at the highest competitive level. All coaches held the highest level of official coaching accreditations with an average of 4.67 ± 3.48 years of experience in an elite head coach role and 11.58 ± 4.80 years of experience in any elite coaching role. The sample was purposively selected from a relatively small eligible population (n = 25).¹⁴ Sample size was similar to studies employing semi-structured interviews of elite coaches.^15,16 Additionally, with difficulty in recruiting elite Australian football head coaches, pragmatism played a role in the final sample size.¹⁷ Coaches were recruited via email and provided written consent prior to participation. Coaches were assured of the voluntary nature of participation and their anonymity and were allowed opportunities to ask any questions prior to the interviews. Ethical approval was obtained from the University Human Research Ethics Committee.

Interviews were conducted by the primary researcher using online video conferencing software (Microsoft Teams IOS v4.14.1 or Zoom v5.10.7, depending on the preferences of the participants). They were recorded using the in-built recording features of the online video conferencing software, and backup recordings were collected with an Apple iPhone 13 pro (Apple, Cupertino, CA, USA). Interviews were transcribed verbatim by the interviewer. Coaches were provided with a full transcript of the interview after transcription to check the transcript for accuracy from which there were no reported concerns with transcript accuracy.

Interview guide

The portion of the interview guide relevant to this study focused on exploring decisions during matches that coaches deemed to be difficult, followed by a shift in focus towards a newly created prototype example of a DSS, specific to Australian football, and what the thoughts and perceptions of coaches were towards the system. Relevant to the first research question, coaches were asked to reflect upon particularly difficult DM scenarios and experiences, with the intent to identify the elements that contributed to that difficulty. Coaches were then asked more directly about any other barriers they face to DM during matches. Stemming from bounded rationality, prompts included: ‘Are there any internal factors that have made it hard for you?’ and ‘Are there any external factors that have made it hard for you?’ To investigate the second and third research questions, coaches were first asked about their attitudes towards being supported by AI, followed by probes, such as ‘If you have any concerns, could you tell me about them?’ and ‘What would you expect it to do?’. This was followed by a brief interactive demonstration using a prototype DSS, where instruction to coaches remained consistent (see Figures 1 and 2(a) and (b)). The purpose of this demonstration was to provide coaches with a material example of what a DSS could look like. The prototype purported to predict end-of-match outcome at different time points during a match and provide suggestions on changes in the magnitude and direction of performance indicators that would lead to an improvement in their likelihood of winning. Coaches were reminded that this was only one potential implementation of a DSS. Finally, follow-up questions to understand their attitudes towards DSSs having seen one potential implementation were asked. The interview guide and prototype were tested in two pilot interviews with sub-elite head coaches. Refinements were made to the DSS interaction portion of the interview by introducing a hypothetical scenario where the coach was guided through how the DSS could be used.¹⁸

Figure 1.

Flow chart of the instructions provided to coaches during the interactive prototype DSS demonstration. DSS: decision support system.

Figure 2.

(a) Screenshots of the prototype DSS. The functionality presented in the above image was the provision of predictions and suggestions. (b) Screenshot of the prototype DSS. The functionality presented in the above image was an interactive simulator, allowing coaches to manipulate values of performance indicators and observe likely changes in win probability. DSS: decision support system.

Analysis

Transcripts were imported into NVivo (NVivo 12, QSR International Pty. Ltd, Australia). The primary researcher analysed transcripts using the six-phase reflexive thematic analysis approach,¹⁹ looking through the lens of bounded rationality and the technology acceptance model.^5,13 Transcripts were read several times and notes were taken on potential themes. Transcripts were then coded reflexively, where inductive and deductive codes were used.¹⁹ For example, some barriers were initially deductively coded as ‘environmental’ or ‘cognitive’ from bounded rationality principles but upon reflection of their over-simplicity, more specific inductive codes (primarily semantic) were used to describe the barrier. Some additional latent inductive codes were used to refer to barriers that were different on face value but related to similar underlying barriers. For example, codes such as ‘coach gets too much information’ and ‘limiting what coaches tell players’ were recoded as ‘cognitive overload’. Themes that represented shared meaning were then generated, reviewed, and refined.²⁰ Themes were reviewed by all named researchers, and some changes were made to names and definitions of themes as well as the groupings of some lower-order themes. Once fully realised, themes were placed into two thematic maps (one for the domain of each research question) and explored in relation to the aims of the study. ‘Critical friends’ were employed to discuss themes and identify opportunities for reflexivity in practice throughout.²¹

Researcher positionality and reflexivity

With the researcher being an active participant in qualitative research methodology,²² it is important to understand the role the researcher's biases and theoretical perspective have on the design, collection, analysis, and interpretation of the data. The overall research project, from which this study stems, purports to understand the DM environment of Australian football coaches during matches to inform best practise for the design and implementation of a DSS. The primary researcher is a doctoral candidate with rationalist epistemological views. The research team also believe there is value in bringing AI into the sport coaching world to improve the translation of sport science research findings to practice. Thus, the data and analyses were guided by the idea that such tools likely could improve the rational DM of coaches.

Throughout the design, data collection, analysis, and interpretation of this study, strategies were employed to mitigate bias and practise personal and interpersonal reflexivity.²³ In the design of the interview guide, the research team discussed the influence that the wording of questions may have on the interviewees, asking ‘are we leading the coach to a response based on our assumptions or encouraging them to think freely?’ Interviewees were made aware of the researchers’ assumptions and views on AI. The primary researcher sought to mitigate the risk of these influencing interview responses by acknowledging this with interviewees and asserting the intent to explore all perspectives on the subject, regardless of these views. Member checking was used to assure that interview transcriptions were an accurate reflection of the interview.¹⁴ Reflecting on the cyclical nature of analysing and interpreting the data was actively encouraged within the research team. Several meetings were held to discuss different interpretations of the data and how preconceived assumptions may be affecting the codes and themes generated.

Results

Barriers to DM

Thematic analysis resulted in the generation of six higher-order themes under two distinct domains. Two higher-order themes for barriers to DM and four higher-order themes for thoughts and perceptions of a DSS are presented. The first sub-section of the results section will focus on the ‘barriers to DM’ domain. Guided by theoretical principles of bounded rationality, two higher-order themes were generated, namely, cognitive and environmental. Seven lower-order themes, each representing a distinct barrier to DM faced by elite Australian football coaches during matches, are presented within (see Figure 3). While some barriers could be considered both environmental and cognitive, a clear distinction was made for the purpose of this study. Barriers stemming from factors internal to the coach's mind were cognitive, while barriers stemming from factors external to the coach's mind were environmental.

Figure 3.

Thematic map, generated through thematic analysis, representing the higher-order and lower-order themes belonging to the ‘barriers to DM’ domain. DM: decision-making.

Cognitive

Cognitive barriers to DM were those that constrained the DM process internally, either as a result of a coach's experiences or cognitive limitations. The ‘cognitive’ theme encompasses barriers such as the coach's emotion, delayed reactions, and misdirected focus. Five coaches discussed at least one cognitive barrier to DM, while one coach did not. Coach's emotion relates to feelings of pressure, unease, or a lack of composure. Four coaches raised coach's emotion as a barrier to effective DM, with one coach suggesting the score can elicit an emotional reaction.

…the emotion of the scoreboard can influence your reaction to things in the moment. (Coach 4)

Similarly, Coach 3 referred to it as ‘the composure barrier’, suggesting that the influence of pressure and composure acts upon a coach's ability to think rationally.

…when pressure comes … it's amazing what pressure does to the brain. (Coach 3)

Despite raising it as a barrier, Coach 5 proposed a solution for moments where emotion develops into a barrier.

You feel nervous, you feel anxiety build and stuff like that. I actually think one of the best forms of decision making is to actually be able to catch your breath and stop … looking at the wall, having a drink… I just find by the coolness of the water going into my pallet gives me a chance to shift my mind away from what I’m actually seeing. (Coach 5)

Briefly pausing and removing themselves from their emotions momentarily to think clearly, Coach 5 employs the strategy to ‘collect [their] thoughts’.

The delayed reaction of coaches is another cognitive barrier that relates to their inability to recognise patterns in the match early enough or acting when it is too late. Three coaches described their experiences relating to this barrier. Seemingly, it was simple to identify in hindsight but difficult to recognise during a match.

…we could have pulled the trigger on that a bit earlier and got a bit more of an ascendancy by pushing [anon player] forward and letting [them] potentially hit the scoreboard before they actually just got so many numbers back. (Coach 2)

…you probably don’t recognise those patterns quick enough, and suddenly you become more reactive. (Coach 5)

Not recognising this during the match could have a detrimental impact on the remainder of the match. According to Coach 4, once this cognitive barrier is experienced, it can be difficult to recover.

I think if you're scrambling game day to chase something then you're probably chasing it and you might not catch it. (Coach 4)

Misdirected focus was the least frequently discussed cognitive barrier to DM. Misdirected focus manifested as an overcommitment or bias to what the coach thought was the most important factor influencing play. By using a single key performance indicator to measure the team's performance during the match, Coach 6 found it difficult to broaden their perspective on the match and led them to act irrationally.

The conversation in the box was almost solely revolving around ‘We said we were going to stop them marking it, they keep marking it, we’ve got to play higher.’ And all the things that should have been the priority in the game took a backseat to this thing that was, in my opinion, a pretty low priority … when the game actually wasn’t going that badly, there was just this one measure that … was given more relevance than it deserved. (Coach 6)

It was suggested this focus towards the wrong aspect of the match could have consequential effects leading to the barrier taking attention from what is most important.

Environmental

Environmental barriers to effective DM during matches were external pressures acting on the coach. They hindered a coach's ability to process information rationally or interact with others involved in DM, either directly or indirectly. Four environmental barriers were identified, with all coaches referring to at least one of the following: time pressure, difficulty communicating, physical environment, and information volume.

Four coaches highlighted that time pressure restricts their DM, affecting their ability to perform in-game analysis, confirm their thinking, and act upon decisions in a timely manner.

Time pressure is the most important thing … linked to time is the ability to make sure that you’re right. (Coach 6)

We’re time poor in game. And letting the coaches think about some of these decisions is critical. (Coach 1)

To an extent, time pressure may result in coaches feeling they need to foresee future match events in order to allow to time to act.

You’ve got to identify things earlier than you probably do and sometimes you’ve got to predict a bit what might occur, and then act on it pretty quickly, because if you wait 6 or 7 minutes … you can’t implement what you needed to at the time. (Coach 3)

Head coaches encountered difficulty communicating as a barrier to DM. It represents a perceived inability for the head coach to efficiently communicate thoughts and ideas efficiently to players due to external factors. This could be affected by time pressure but is considered a distinct barrier as it is also constrained by communication channels (i.e., the runner) during the match and a breakdown in understanding by the players as a message is communicated throughout the team. As a result, four coaches expressed that they may identify potential options but refrained from exploring them further or implementing them due to concern that their communication of the message would be ineffective.

…[discussing why they hesitate to make a decision], that communication piece, if our forwards then don't react and go into the mode that we want them to knowing there's going to be an extra, I just think it creates a disjointed system and you're not connected. (Coach 4)

…what are the chances of what we’re proposing to do succeeding, as opposed to kind of the reverse, which is: This would be fantastic if we could get this exactly right, but the chances of getting that message through are low. (Coach 6)

According to four coaches, the physical environment is a barrier to DM. Interestingly, two of the coaches referred the physical environment in relation to where they were physically located (at ground level) and the affect it had on their ability to perceive the performance of players and or teams.

When you're at ground level, you lose the ability to have an overview of what's happening. (Coach 4)

Meanwhile, the other two coaches referred to the physical environment in relation to the people surrounding them. They raised that without a ‘collaborative, calm environment’, their ability to make decisions in a rational and collaborative way is hindered. In cases where such an environment is not set, they felt they were at risk of making incorrect decisions.

I don’t think you can make clear, precise, correct decisions, or even discuss them and arrive at an outcome unless you set the right environment. So, I think the environment is really, really important and can be either a barrier or a bonus if you get it right. (Coach 2)

The final environmental barrier, information volume, relates to the amount of data available and visible to coaches during matches. Four coaches discussed their preferences for a pared-back approach to receiving data during a match, suggesting that an overabundance of information can hinder their ability to consult and monitor the information that they considered most important.

I still think we could have ripped half of those numbers out and been a lot clearer and not reacted to too much more. (Coach 3)

There's a lot going on, so I think [too much information is] probably a little bit of a barrier as well. When you’ve got so much going on and just trying to remove some of the information… (Coach 2)

Meanwhile, two coaches were content with the volume of information. Rather suggesting that it is ‘up to the skill of the coach to filter through that stuff and get what he needs’. Coach 6 also suggested there was perhaps room for more information to be collected and communicated.

I just ask myself ‘what is it we don’t know that we haven’t even thought about?’. That's the part that sort of leaves me a little bit anxious. It's one thing having the information to work through, but is there something that's really important that we haven’t even thought of? (Coach 6)

Thoughts and perceptions of a DSS

Four higher-order themes were generated in this domain, namely willingness to adopt a DSS, expectations, concern and criticism, and the role of the DSS. Within these, eight lower-order themes relating to thoughts and perceptions of a DSS were generated (see Figure 4).

Figure 4.

Thematic map, generated through thematic analysis, representing the higher-order and lower-order themes belonging to the ‘thoughts and perceptions of a DSS’ domain. DSS: decision support system.

Willingness to adopt

When discussing their attitudes towards having their decisions supported by AI during matches, all coaches exhibited intrigue and eagerness, while four coaches also raised potential encounters with feelings of replacement or hesitancy to adopt a DSS by their support staff. All coaches were open to having their subjective assessments supported by AI, with all suggesting a growing trend towards the utilisation of AI. Adding to this, Coach 6 suggested that early adoption would provide an opportunity to manage flaws in any new systems as soon as possible.

One thing I’ve got 100% conviction on is that we will use data, algorithms, whatever you like, insert here, AI, objective information, overlayed with intelligence much more than we do now. So, my theory is kind of get with the program and deal with things that might not work perfectly. (Coach 6).

Unsurprisingly, despite their personal intrigue and eagerness, four of the coaches expected that some support staff may be hesitant to adopt AI. Performance analysts, sport scientists or assistant coaches may feel as though AI is being introduced to replace them.

It might mean that some people are threatened by the fact that their role might become obsolete. But I think it's important. (Coach 1)

Coach 4 suggested this is due to many people in the industry being ‘set in their ways’. Meanwhile, Coach 1 highlighted this could be due to someone feeling like ‘their potential to grow and develop might be diminishing’. Despite this, all coaches were welcoming of the introduction of AI-based decision support, though cognisant of the need to introduce it in a way that does not alienate their assistant coaches, performance analysts, and sport scientists.

Expectation

All coaches expressed thoughts on their expectations of a DSS. In doing so, they discussed several elements that were related to the ease of use of a DSS and what the coach wanted from the DSS. Four coaches discussed the use of colour and/or size being important due to the impact it has on quickly drawing the attention of coaches to matters of importance.

…colour hits you straight away, so that's where my eyes go to straight away, and then the bigger it is, obviously that attracts your eyes as well. (Coach 5)

Building on this, three coaches expressed the importance of simplicity in the visual design, navigation, and language of a DSS.

If it's simple for us, we can make it simple for the players. Whereas if it's complicated for us, I think that's when you can make it complicated for the players. (Coach 4)

Coach 1 added that some coaches use different ‘coaching language’, and therefore, the language used in the DSS should account for variations in the terminology used by different coaches.

What coaches want refers to the coach's expectation of value or usefulness of a DSS. All coach's referred to the ability of a DSS to provide support in terms of problem recognition, diagnostics, and prediction. Coach 1 highlighted the potential value of having ‘triggers through statistics that would spark some of the conversation’. Meanwhile, Coach 2 added they would benefit from a DSS identifying important areas of the game that needed improvement during a match and providing suggestions on the areas of the game requiring the most attention.

[I’d like to see] … an algorithm that says you’re winning this, you’re winning that, but you’re losing that right? So, you gotta fix up that part of the game and not worry about the other parts. (Coach 2)

This sentiment was echoed by Coaches 3, 5, and 6, with Coach 3 emphasising the importance of ‘having the right information’ to ensure they address the most important aspects of the team's performance. It was proposed by Coach 6 that being able to predict match events using data would be useful in supporting a coach's intuition.

In the moment, how can we see a shift in the way the game is played through data before the scoreboard actually has an impact?… you can sense the change…, but it would be nice to have data to support it. (Coach 6)

Coach 6 also discussed the value in being able to quickly analyse team performance without reference to the score. It was suggested that a counterintuitive perspective could support the decision not to make any changes to the way the team is performing.

It's the counterintuitive stuff that I’m interested it … we’re 4 goals down, but the data is showing us a change here might be counterproductive… We could flip our team around and change our strategy that we spent so much time on going into the game, or we can identify that these are outlier events… Aside from trying to halt the momentum a bit and stopping the team scoring for a couple of minutes, we don’t want our best laid plans to go out the window. (Coach 6)

Concern and criticism

Despite being mostly receptive of the implementation of a DSS during matches, some coaches voiced concerns and criticism of a DSS. Two coaches were concerned over a potential for overreliance on a DSS, which could ultimately lead to diminished faith in the coach's expert abilities and intuition.

I've seen some coaches that rely too much on numbers and then lose touch with what's the human element of the game. (Coach 4)

Coach 6 also added that people could lose faith in a DSS if it doesn’t ‘work perfectly right from the start’.

Four coaches were concerned about the potential over simplicity of a DSS. The concern surrounded the use of simple measures of performance, which may seem obvious to a coach.

…if I said to my coaches, ‘Hey, with these metres gained, we win the second half, if we get that to 800 metres gained, it's going to help us win the game,’ and they’re going to go ‘F**k, no shit Sherlock… How do we do that?… How are we then going to do that if we’re playing against [anon team]?’ (Coach 3)

Some coaches were also concerned about relying on generalised decision support, which may not capture the specific play style of different teams. Coach 5 highlighted that they were not interested in knowing the ‘competition average’ but would rather know specific information relating to their team. Coach 3 agreed, suggesting that decision support needed to be team-specific.

…the opposition have different strengths and weaknesses as well. Every team plays slightly differently. If you play against [anon opposition team] there's a different statistic, or a different group of KPIs you need to look at, although they’re very similar. (Coach 3)

DSS role

All coaches speculated on the role they expect the demonstrated DSS to play during matches. This resulted in a discussion of how and when a DSS would be used, as well as who would use it. Five coaches expressed a similar sentiment regarding when a DSS would be used the most, pointing towards moments in which their team is being beaten or has experienced a negative shift in momentum.

I think it's when we need change, we’ll go to it, or the momentum in the game has shifted and there's a lot of aspects of the game that are going against us. (Coach 1)

Coach 6 added that it would be most useful to have a DSS that could update regularly throughout a match to provide support at any time as the time to do so during breaks (quarter-time, half-time, and three-quarter-time) is insufficient.

If it's just at the breaks, I think it would need to be done really quickly at the start of the break. The time pressure just becomes too much, especially at the quarter breaks. (Coach 6)

In addition to when it would be used, some coaches discussed how they would use a DSS. Two coaches suggested that they would use it to ‘spark conversation’ and be ‘thought provoking’. Meanwhile, Coaches 3 and 6 imagined using a DSS to not only provide objective support for their subjective views but also support them in exploring the likely success of potential options.

We would first throw up a hypothesis based on what we see. Make sure the data supports it, come up with a working theory, which predictions might help. If that all lined up, we’d move to the simulator page and say ‘if we improve this metric by 25%, that would be enough to win the game. Let's move forward on whatever the strategy is to try and achieve that aim’. (Coach 6)

All coaches agreed that while a DSS is intended to support the DM of the coach, it would be an analyst in control of the system. The analysts would alert and feed information to the coach when necessary and interact with the DSS; however, coaches may also be able to see the DSS, either on a monitor in the box or on their own device.

Our analyst would use it the most… I could see them monitoring that in real time and then the conversation happening when something significant changes, or at the breaks. (Coach 6)

Our analysts would use it. If we were to bring something like this in it would also be displayed on a screen for us to see, and it would be something we’d refer to regularly. (Coach 3)

Discussion

This study aimed to identify the existing barriers to effective DM for Australian football coaches during matches and the thoughts and perceptions of coaches towards AI-based DSSs. The intention was to improve understanding of the specific barriers that constrain DM in this context and how coaches would feel about the introduction of AI-based solutions to any barriers. The notion of an Australian football coach's DM being hindered by cognitive and environmental constraints was supported by the barriers identified in the thematic analysis. Further, the thoughts expressed by coaches about using AI to support their DM presented a range of perspectives and valuable insights into how a DSS could be most effectively designed and implemented to minimise some of these barriers.

Several barriers to effective DM, more generally, have been identified in the psychology literature, including time pressure, information overload, uncertainty, decision complexity, and emotion.^24,25 Limitations to human information processing speeds may underpin several of these barriers.⁴ Unsurprisingly, some of these barriers were perceived by Australian football coaches, though the specificity of how these barriers present in an Australian football context is worth understanding. For example, time pressure, like in most team sports, is inevitable when coaching Australian football due to its fast-paced nature. However, knowing coaches felt the need to ‘predict’ future events to ensure time to implement a change effectively deepens understanding of the impact of time pressure on DM in this context. This should prompt future work on methods to support the foresight of coaches. Similarly, sports-specific opportunities arise from an understanding that the ‘emotion of the scorecard’ can negatively influence DM. Score margin has been shown to impact coach verbal behaviour, the amount of feedback given to players during matches, and now DM.^26,27 It could be suggested that methods for reducing the impact or weight of the match score on DM are of value when designing and implementing a DSS. Importantly, none of the barriers seemed to be entirely isolated from another, meaning there may be an interaction between each barrier and at least one other. For example, time pressure when making an important decision could be a key contributor to the delayed reaction of a coach. Consequently, any solution to minimise the impact of these barriers on a coach during matches be designed to target multiple barriers, rather than one in isolation.

The information volume barrier raises an interesting point on the possible assumption that too much information is always the concern. Instead, it was suggested that some coaches would prefer to have less information; meanwhile, some would prefer to have access to more information. In the context of constraint minimisation for Australian football coaches, this highlights the importance of understanding that a ‘one-size fits all’ approach may not be appropriate and emphasises the importance of including end-users in the design of new technologies.²⁸ Perhaps a DSS requires some flexibility, allowing coaches to access more information if they desire but not as a default.

The exploration of thoughts and perceptions towards AI-based DSSs provided insight into important considerations for the design and implementation of an AI-based DSS. Specifically, a greater understanding of the willingness of coaches to adopt AI, concerns and criticism, expectations, and the role of AI can guide the development and implementation of such tools. To address the barriers that could feasibly be affected by an AI-based DSS, the system must be appealing to those whom it purports to benefit. Many of the lower-order themes support the broader perspective proposed by the technology acceptance model, in that if they are to be accepted, new technologies must be perceived as useful and easy to use.¹³ Despite this, the range of attitudes towards willingness to adopt, with coaches expressing eagerness and hesitancy, suggests there will be differing perspectives amongst users regardless of ease of use and usefulness, perhaps due to the user's personal traits, knowledge, and understanding of AI.²⁹ As coaches proposed that analysts would use the tool, yet the aim of such a tool is to support the DM of coaches, there could be difficulty in translating information from the DSS to coaches if the system design is just targeted towards coaches. Solely focussing on achieving the ‘buy-in’ of coaches may represent ignorance of the function of coaching teams where analysts, sport scientists, and coaches all have a role in DM. Thus, efforts at achieving ‘buy-in’ should be made with respect to coaches as well as all other support staff. Doing this could help to achieve integrated intelligence, by which AI is complimentary to DM the system and users are reciprocally interdependent.³⁰

Coaches expect for a DSS to be easily navigable, use colour to highlight important information, and be easy to understand. Meanwhile, they expect it to provide added value through the provision of data-driven insights that may counter or confirm what they are already seeing or thinking. The need to facilitate this advances the idea that the design of the DSS should be customisable and should undergo some user-centric co-design.^28,31 Throughout this process, concerns such as over-simplicity and over-reliance could be addressed or dispelled. For example, while a DSS could appear to be overly simple in terms of the metrics used or advice given, the purpose of a DSS is to provide access to complex systems in a simplified manner.⁷ Through the co-design process, some of the underlying complexity of the system can be explained to coaches and breakdowns of seemingly simple metrics could be provided. Allowing users to be involved in the design process may provide them with a deeper understanding of the complexities involved in presenting the eventual output. Additionally, the inclusion of specific opposition-based information could be important in improving the perception that a DSS is complex. Concerns of overreliance could be addressed by showing transparency through representations of uncertainty in the confidence of AI-based suggestions and teaching users to understand the meaning of these representations.³²

Knowing when and how an AI-based DSS would be used highlighted an interesting, context-specific insight. Coaches prefer a real-time system to be capable of in-play, regular live updates due to the lack of time available for additional analysis during quarter breaks. With consideration of the ‘time pressure’ barrier to effective DM, this is unsurprising. However, it does add additional complexity to the technical development of a DSS. Schelling and Robertson³³ proposed that a DSS developed for high-performance sport should consider context satisfaction, output quality, and process efficiency. Thus, there is a need to ensure that improvements in context satisfaction such as providing live, in-play updates do not come at the expense of output quality or process efficiency. Further research to investigate the trade-off between these considerations may be necessary.

While a DSS could be useful for reducing the impact of some barriers, it is important to note that not all barriers identified could feasibly be improved upon by a DSS and may require alternative solutions or considerations. Physical environment barriers relating to coaching team dynamics in the coaching box may be improved by building trust and setting expectations with support staff pre-competition. Additionally, an understanding between coaches and support staff about their individual coaching philosophies could improve these team dynamics.³⁴ Meanwhile, the decision to coach from ground level could be seen as a trade-off, choosing to minimise difficulty communicating with players, sacrificing the ability to have a strategically advantageous view of the match. The decision to make this trade-off could relate to trust in assistant coaches and communication skills within the coaching team, though understanding the motivations behind a decision to accept this trade-off could be a topic of future investigation.

This study is the first to identify the barriers Australian football coaches face when making decisions during matches and being the first exploration of Australian football coaches’ attitudes towards AI-based DSSs. Additionally, while some literature has explored attitudes towards new technology in sport coaching more broadly,^31,35 this study specifically explores in-match AI-based implementations with a sample of coaches who are likely to have access and funding to support this technology (i.e., professional). The findings may exhibit naturalistic generalisability for elite sport coaches in sports outside of Australian football.³⁶ Coaches and practitioners with access to similar resources and support staff can reflect on their own experiences and determine whether the barriers raised by Australian football coaches exist in their domain as well. Moreover, anyone intending on developing and implementing an AI-based DSS for real-time use in elite sport may find that the barriers and attitudes discussed in this paper to be transferable to specific processes and practices in their own coaching environment. Despite this, there are some important limitations to note. The findings relating to DM barriers are dependent on the coach's recall of difficult DM experiences and consequentially reflect what coaches perceive to be the barriers they face. Longitudinal research designs involving barrier minimisation and follow-up interviews may be necessary to confirm the impact of the barriers on DM. Findings relating to attitudes towards using a DSS may also suffer partly from biases introduced by the presentation of a prototype DSS. While it was an important method of helping coaches understand possible implementations of a DSS, it was only one type of DSS and could have potentially constrained their thinking about what a DSS is and what it can be used for. Relevantly, an understanding of researcher positionality is important to interpreting and understanding these findings. Finally, the coaches opting to participate in this study were aware of the focus in some parts on AI-based DSSs, meaning it is possible the coaches interested in participating were more open to AI-based innovations than those who were invited to participate but did not.

Conclusions and practical implications

This study has identified seven barriers to the effective DM of elite Australian football coaches during matches and has provided some insight into the thoughts and perceptions of these coaches towards the implementation of AI-based systems to support DM. Coaches are constrained by several environmental and cognitive barriers, some of which may be supported by AI and some of which may not. Elite Australian football coaches are also open to the implementation of AI-based DSSs; however, caution should be taken in ensuring the system is easy to use, transparent and trustworthy, and guided by the requirements of the coaching team. The practical implications of these findings are that AI-based DSSs may be useful for supporting coaches in alleviating time pressure, improving slow reactivity, managing information volume, reducing the impact of emotion, and directing focus to important areas of performance. Further, researchers and practitioners should gain an understanding of the expectations and concerns of coaches and support staff through co-design to best facilitate the design and implementation of such systems.

Footnotes

Acknowledgements

The researchers would like to thank the coaches and their respective football clubs who gave their time to participate in this study. We value and appreciate your input.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship, and/or publication of this article.

ORCID iDs

Mitchell F Aarons

Lyndell Bruce

References

Smith

Cushion

. An investigation of the in-game behaviours of professional, top-level youth soccer coaches. J Sports Sci 2006; 24: 355–366.

Aarons

Vickery

Bruce

, et al. A framework to explain the in-match decision-making of elite Australian football coaches. Psychol Sport Exerc 2023; 67: 102439.

Robertson

. Man & machine: adaptive tools for the contemporary performance analyst. J Sports Sci 2020; 38: 2118–2126.

Taraban

. Limits of neural computation in humans and machines. Sci Eng Ethics 2020; 26: 2547–2553.

Simon

. Bounded rationality in social science: today and tomorrow. Mind Society 2000; 1: 25–39.

Robertson

. Man & machine: adaptive tools for the contemporary performance analyst. J Sports Sci 2020; 38: 2118–2126.

Sprague

. A framework for the development of decision support systems. MIS Q 1980; 4: 1–26.

Robertson

Bartlett

Gastin

. Red, amber, or green? Athlete monitoring in team sport: the need for decision-support systems. Int J Sports Physiol Perform 2017; 12: 73–79.

Calder

Durbach

. Decision support for evaluating player performance in rugby union. Int J Sports Sci Coach 2015; 10: 21–37.

10.

Wright

Atkins

Jones

, et al. The role of performance analysts within the coaching process: performance analysts survey ‘The role of performance analysts in elite football club settings’. Int J Perform Anal Sport 2013; 13: 240–261.

11.

Watson

Hendricks

Stewart

, et al. Integrating machine learning and decision support in tactical decision-making in rugby union. J Oper Res Soc 2021; 72: 2274–2285.

12.

Windt

MacDonald

Taylor

, et al. “To tech or not to tech?”’ A critical decision-making framework for implementing technology in sport. J Athl Train 2020; 55: 902–910.

13.

Davis

. A technology acceptance model for empirically testing new end-user information systems: theory and results . PhD Thesis, Massachusetts Institute of Technology, 1985

14.

Green

Thorogood

. Qualitative Methods for Health Research. London, UK: Sage Publications, 2018.

15.

Mouchet

Harvey

Light

. A study on in-match rugby coaches’ communications with players: a holistic approach. Phys Educ Sport Pedagogy 2014; 19: 320–336.

16.

Mason

Farrow

Hattie

JAC

. Sports coaches’ knowledge and beliefs about the provision, reception, and evaluation of verbal feedback. Front Psychol 2020; 11: 2424.

17.

Braun

Clarke

. To saturate or not to saturate? Questioning data saturation as a useful concept for thematic analysis and sample-size rationales. Qual Res Sport Exerc Health 2021; 13: 201–216.

18.

Turner

III . Qualitative interview design: a practical guide for novice investigators. Qual Rep 2010; 15: 754.

19.

Braun

Clarke

Weate

. Using thematic analysis in sport and exercise research. In: Smith B and Sparkes AC (eds) Routledge handbook of qualitative research in sport and exercise. London: Routledge, 2016, pp.191–205.

20.

Braun

Clarke

. Reflecting on reflexive thematic analysis. Qual Res Sport Exerc Health 2019; 11: 589–597.

21.

Smith

McGannon

. Developing rigor in qualitative research: problems and opportunities within sport and exercise psychology. Int Rev Sport Exerc Psychol 2018; 11: 101–121.

22.

O’Sullivan

Jefferson

. A review of strategies for enhancing clarity and reader accessibility of qualitative research results. Am J Pharm Educ 2020; 84: 147–155.

23.

Olmos-Vega

Stalmeijer

Varpio

, et al. A practical guide to reflexivity in qualitative research: AMEE Guide No. 149. Med Teach 2023; 45: 241–251.

24.

Phillips-Wren

Adya

. Decision making under stress: the role of information overload, time pressure, complexity, and uncertainty. J Decis Syst 2020; 29: 213–225.

25.

Lerner

Keltner

. Fear, anger, and risk. J Pers Soc Psychol 2001; 81: 146–159.

26.

Mason

Farrow

Hattie

JAC

. An analysis of in-game feedback provided by coaches in an Australian Football League competition. Phys Educ Sport Pedagogy 2020; 25: 464–477.

27.

Gómez

Luján

JFG

Santamaría

. Relationship between score and coaches’ verbal behaviour. J Hum Sport Exercise 2013; 8: 728–737.

28.

Trischler

Dietrich

Rundle-Thiele

. Co-design: from expert- to user-driven ideas in public service design. Public Manage Rev 2019; 21: 1595–1619.

29.

Duan

Edwards

Dwivedi

. Artificial intelligence for decision making in the era of Big Data – evolution, challenges and research agenda. Int J Inf Manage 2019; 48: 63–71.

30.

Lichtenthaler

. Extremes of acceptance: employee attitudes toward artificial intelligence. J Bus Strategy 2020; 41: 39–45.

31.

Ringuet-Riot

Carter

James

. Programmed innovation in team sport using needs driven innovation. Procedia Eng 2014; 72: 817–822.

32.

Schmidt

Biessmann

Teubner

. Transparency and trust in artificial intelligence systems. J Decis Syst 2020; 29: 260–278.

33.

Schelling

Robertson

. A development framework for decision support systems in high-performance sport. Int J Comput Sci Sport 2020; 19: 1–23.

34.

Hall

Gray

Sproule

. The microstructure of coaching practice: behaviours and activities of an elite rugby union head coach during preparation and competition. J Sports Sci 2016; 34: 896–905.

35.

Jaswal

Stergiou

Katz

. Attitudes and behaviours of Canadian National Team coaches regarding the use of technology in their coaching practice. J Hum Sport Exercise 2019; 14: S2445–S2454.

36.

Smith

. Generalizability in qualitative research: misunderstandings, opportunities and recommendations for the sport and exercise sciences. Qual Res Sport Exerc Health 2018; 10: 137–149.