Can machine learning be used to forecast the future uncertainty of military teams?

1.1. An overview of uncertainty

Uncertainty means new information has been detected. It has its roots in the mismatch between arriving sensory information and a person’s expectations of the world based on their prior experiences. ⁵ Through a process initially involving thousands of neurons, the ambiguities between the mental models and incoming information are resolved by collective neural decisions as uncertainty is propagated towards consciousness, ⁶ and then further resolved by collective interactions in the social realm. ⁷

Uncertainty is described at the brain level as being sensory, state, rule, or outcome based, ⁹ and while the orbital prefrontal cortex seems a locus for uncertainty-related activity, especially for outcome and sensory uncertainty, ¹⁰ other active areas exist across the brain. Uncertainty monitoring shows parallels with metacognition and may or may not require mental workload depending on the experience of the subjects. ¹¹

Despite the ubiquity of uncertainty, and our ability to recognize it, the dynamics of uncertainty have been difficult to measure within the complexity of real-world tasks. This is due, in part, to uncertainty being based on each person’s experiences, which guides their perceptions of novel information—information that needs to be organized, identified and interpreted in order to understand its significance. ¹² What might be highly uncertain for one person might be unimportant or not even perceived by others.

1.2. Neural correlates of team uncertainty

Of the myriad of time frames between the implicit firing of neurons and periods of hesitation ¹³ or anxiety ¹⁴ , we have developed systems for quantitatively measuring uncertainty over seconds to minutes.^15-18 This time frame is greater than the 100–300 ms neurodynamics associated with everyday processes like conversations and non-verbal interactions,^19-22 and is optimized to identify periods of persistent uncertainty that could be mitigated if detected early.

Persistent neurodynamic organizations can be thought of as complex extensions to the momentary neuronal pauses recorded from the pre-motor cortex, as in monkeys during periods of behavioral hesitation and indecision. ²³ Observationally, these pauses might go unnoticed or appear as periods of silence, ²⁴ but neurodynamically the pauses indicate that irregularities have been detected that require expanded and extended brain resources to resolve—i.e., there is thus the need to expend more energy.^25, It is not surprising, therefore, that submarine navigation teams with higher levels of neurodynamic organization during submarine navigation simulations were rated lower by their instructors than those with lower levels. ²⁶

To better understand the challenges of teaching AI systems about human uncertainty, we recently trained machines to recognize neurodynamic organizations associated with verbalized uncertainty. ²⁷ While primarily trained to recognize uncertainty by students performing a wayfinding task, there were indications that the trained neural networks might identify uncertainty patterns arising during more complex teaming situations.

The hypotheses being tested in this study were that:

2.1. Task

Two electroencephalogram (EEG) datasets were used in these studies. The first dataset was from pre-college and college students performing a wayfaring task called the Map Navigation Task (MT).^15,28 This is a cooperative task chosen because of its ability to induce uncertainty in the participants. Two persons face each other while viewing a computer-displayed map with multiple landmarks. The two maps are similar but not identical, and subjects cannot see each other’s map. The instruction giver (Giver, abbreviated [G]), has a printed path through the landmarks and verbally guides the follower (Follower, abbreviated [F]) in duplicating that path. The neurodynamics of MT performances are presented in this paper as an illustration of how neural data streams containing periods of uncertainty can be parsed to reveal the scalp locations and EEG frequencies involved during uncertainty.

The second dataset was obtained from experienced navigation teams at the US Navy Submarine School, Groton, CT who performed Submarine Piloting and Navigation (SPAN) simulations as part of their standard training while wearing EEG headsets.^16-18,29 The two SPAN simulations described were performed by an experienced navigation team practicing establishing and maintaining the ship’s position while in close proximity to other ships and the land. This EEG dataset served as a test set to evaluate the ability of an AI system trained with exemplars of pre-college students’ uncertainty to generalize to the identification of uncertainty with a complex task.

SPAN sessions contained three training segments: Briefing; Scenario, and Debriefing. In the Briefing, the team reviewed the environmental conditions and ship traffic and statically established the submarine’s position. In the Scenario, the team dynamically avoided encounters with other ships and shoals, and managed instrument failure and changing weather.

A repeated team process updated the ship’s position every three minutes. In this process, called ‘Rounds’, three navigation landmarks were chosen and their visual or electronic bearings from the boat were measured and the chart updated. The regular Rounds sequence began with a ‘1 min to next round’ call followed by a ‘mark the round’ call 60 s later. The ‘Mark Round’ segment is where data obtained during individual tasks, such as reading the fathometer or radar, became teamwork as the information was shared. Rounds were completed when the ship’s position was charted and the call to ‘end round’ was made. The Debriefing was an after-action review where all team members participated in critical performance discussions.

The four crew members fitted with EEG headsets were the Navigator (NV), the Officer on Deck (OD), who was in charge of the ship for their watch, the Contact Manager (CM), who kept track of other ship traffic, and the Quartermaster (QM), who maintained the ship’s position during their watch (other people were ‘satellite’ team members but were not directly involved in the team processes analyzed here).

2.2. Subjects

Informed consent protocols were approved by the Biomedical IRB, San Diego, CA (Protocol EEG01) and the US Navy Medical Review Board for the collection of EEGs from the submarine navigation team. All participating subjects consented to participate with written approval, and to make their images and speech available for additional analysis. To maintain confidentially, each subject was assigned a unique number known only to the investigators of the study, and subject identities were not shared. This design complies with DHHS: protected human subject 45 CFR 46; FDA: informed consent 21 CFR 50.

2.3. Preprocessing EEG signals

The data acquisition began shortly after the EEG sensors were adjusted for good contact (<10 Ω) and the team member data streams and audio and video recorders were synchronized with electronic (e.g., timing and latency < 2 ms) markers. The recorded EEG data pre-processing used separate high and low pass frequency filters and robust average referencing ³⁰ for detrending the data in order to properly calibrate the thresholds prior to detecting and adaptively removing sinusoidal noise ³¹ using the Matlab^®-based FieldTrip^® toolbox ³² or the open-source EEGLAB signal processing environment, ³³ as detailed previously.^17,34 Commonly found artifacts are generated from speech, eye blinks, heartbeats, breathing rhythms and other electromyography sources. As neurodynamic organizations regularly occur during silence, speech is an unlikely source for most organizations.^15,16 Power spectral density (PSD) values were developed using the Welch method ³⁵ for each frequency band of the 1–40 Hz frequency spectrum. These preprocessing steps can be performed in real-time using software like Neuropype (Intheon, Inc.).

Commercial EEG headsets with both dry and wet electrodes from multiple vendors were used, with the number of sensors ranging from 5 to 19. A greater number of sensors allows more detailed analysis of the spatial locations of the sources of uncertainty in the brain.

2.4. Modeling neurodynamic organizations and information

The next modeling steps used the preprocessed EEG signals to develop spatial and temporal neurodynamic symbols from the PSD values of each team member. This information is illustrated in Figure 1.

OPEN IN VIEWER

Figure 1.

Team and individual neurodynamic modeling of a dyad. (a) A sample Neurodynamic Symbol (NS) showing a 1 s period where the EEG power was high for the Quartermaster (QM) and average for the Officer on Deck (OD). (b) The nine-symbol Neurodynamic State Space (NSS) for two persons with three EEG power levels. (c) The distributions of the -1, 1, and 3 symbols need to be the same number for accurate quantitative comparisons. (d) Neurodynamic Data Streams (NDS) are symbol sequences that span the performance. For a dyad, they are the symbols in Figure 1(b). For team members they were the -1, 1, and 3 values used symbolically. Note that the symbol expression for both team and individual NDS were not random but punctuated by periods of symbol repeats. (e) Variations in the distribution of symbols in the NDS are measured as bits of information within a 60 s sliding window. The fewer NS expressed within a window, the greater the number of bits of information.

The modeling first separated the EEG PSD levels each second into performance average high, medium, and low power levels and assigned them the symbols 3, 1, and -1 (i.e., upper third, middle third, lower third), respectively. These could be any symbol, and there could be any number of categories, and while we have used from 3 to 12 levels, 3 levels allow more understandable visualizations to be constructed (Figure 1). The entire performance of any team member could then be described by symbol streams of -1, 1, or 3’s (Figure 1(b)). The data from each team member (shown for a dyad in Figure 1(a)) was combined into a composite symbol representing the team state. With two persons and three states, the team symbolic history consisted of data streams of the symbols 1 to 9.

The entropy of the neurodynamic symbol streams can never be greater than the maximum of the number of symbols: for three PSD levels, these would be 3.17 bits for a 9 symbol dyad, or 1.59 bits for an individual. The entropy can also never be lower than zero, which is the entropy of a single symbol. These mathematical limits mean that the entropy of any team of two persons where the EEG is separated into three levels will have entropy levels between 0 and 3.17 bits.

The detection and quantitation of uncertainty and hesitation used information modeling over windows of 60 s that were updated each second. The entropy ³⁶ of these 60 s segments was calculated, and then subtracted from the maximum entropy for the number of unique symbols, giving Neurodynamic Information (NI), which is the positive number of bits of information each second.

3.1. Parsing and visualizing the temporal and spatial neurodynamic information flows within a dyad

The temporal and spatial flows of neurodynamic information are first illustrated for a dyad that performed a scenario of the Map Navigation Task (Figure 2). This is presented as an example of the dynamic flow of information throughout the brain and across members of a team as uncertainty is experienced.

OPEN IN VIEWER

Figure 2.

Quantitative comparisons of NI across spatial scales. (a) The average bits of information of the dyad, Giver and Follower, and their shared information is shown for a Map Task performance; the vertical lines help show peak alignment. (b) The NI dynamics are plotted over time at each sensor of the Follower. The Mouse Clicks bar to the right shows the density of his mouse clicks, which increased around 330s when he experienced difficulties drawing the path. (c) The elevated NI at the C3 sensor is expanded for one segment (330s–390s) and displayed across the 1–40 Hz EEG frequency spectrum. (d) A profile plot of NI in the 18 Hz frequency band is overlaid with a bar plot of the -1, 1, and 3 EEG-PV. All NI values are shown after the subtraction of NI from parallel randomized NDS.

During this performance, the Giver (G) was verbally directing a second person, the Follower (F), in drawing a line through landmarks on (F’s) computer screen. During the task, the team was exchanging information, and (F) was drawing paths with the computer mouse around the task landmarks. Around 350 s, (F) had difficulties drawing the path with the computer mouse cursor, and in an attempt to gain control while uncertain of why the difficulty occurred, (F) began rapidly clicking the mouse to send commands (Figure 2(b)). As the Follower became frustrated (indicated by (F’s speech), the NI first increased in the parietal region (P0z, P3) and subsequently to the pre-motor/motor region (C3 and C4 sensors).

This figure illustrates the quantitative functionality of the information algorithm by tracking the NI enrichment for each step of the analysis. For the Follower (Figure 2(a)), the average NI was 0.035 bits, which increased to an average of 0.16 when measured at the C3 sensor and 0.26 bits when measured at the 18 Hz frequency band of the C3 sensor (Figure 2(d)). The peak NI value was 0.46 bits or ~30% of the maximum possible for 3 symbols (1.57 bits).

Lastly, a likely cause for the elevated NI levels during the 70 s segment in Figure 2(d) could be identified by comparing NI and EEG power values (EEG-PV). The bar graph in the lower part of Figure 2(d) shows that the mean EEG power value was 0.26 for the 70 s segment, which was significantly lower than the expected average (1.0) for equal numbers of symbol -1, 1, and 3 values (Z= -2.97, p < .05). The combination of high NI and low EEG-PV in the sensorimotor (C3) region suggests this was a period of persistent deactivation of 18 Hz mu rhythms, which occurred when F attempted unsuccessful drawing movements. ²¹

The following features of the information algorithm illustrated in Figure 2 show the depth of information that can be determined during education and training simulations.

This example shows how an instructor (or trainee) using an AI system built around this information algorithm can choose to have neurodynamic uncertainty reported (a) for cohorts of different teams, (b) for team members functioning in the same or different positions, (c) for specific training segments, (d) for specific periods of increased (or decreased) motor activity, executive function, etc.

3.2. SPAN team information flows during the Rounds sequence

The above NI algorithm was applied to the EEG signals of the NV, QM, and CM who were part of the control room navigation team (Figure 3) and begins after the FzP0 EEG channel had been identified as one with the highest NI levels. The 11 Hz activity is highlighted as the peaks at this frequency were more discrete than in other EEG bands thus allowing a better estimation of the duration of each peak.

OPEN IN VIEWER

Figure 3.

Visualizing the NI (left) and EEG-PV dynamics (right) of the navigation team. (a) The countdowns for the last minute of the five Rounds sequences are plotted vs. time. (b, c, d) These figures plot the NI for the Navigator (NV) Quartermaster (QM) and the Contact Manager (CM). (e, f, g). These figures plot a centered 60 s moving average window of the EEG-PV from the 11 Hz frequency. All values are from the FzP0 dipole of each person.

The countdown events for the final 1 minute of the Rounds sequence (i.e., 1 minute to go, 30 s to go, etc.) are shown in Figure 3(a), and the largest NI peaks were all located during this 1 minute period. The performance-averaged NI values ranged from .08 bits for the QM to .14 bits for the NV and were weakly correlated with each other (NV-QM, r =.36; NV-CM, r = .3; QM-CM, r = .34; p < .01 for all), but not synchronous.

The EEG-PV fluctuations at 11 Hz were next modeled for the three team members using the -1, 1, and 3 symbols numerically and aligning them with the NI and Rounds events using a 60 s moving average (Figures 3(e–g)). The EEG-PV peaks fluctuated around the mean value of 1 and showed sustained periods of both deactivation and activation. Visually the peaks were less discrete than those of NI, and there was no correlation between NI and EEG-PV (average r = -0.06 across the three team members.

These results indicate that the information algorithm principles outlined in Figure 2 apply to more complex activities like submarine navigation. The next sections use the neurodynamics of this navigation team as a testbed for probing an AI system’s understanding of the behavior of uncertainty.

3.3. An AI system’s perspective of navigation uncertainty

Recently we trained artificial neural networks to recognize pattern variations in the NI peaks associated with verbalized uncertainty ²⁷ using self-organizing maps (SOM). SOM is an unsupervised architecture where during repeated presentation of exemplars, a topology is generated such that similar vectors become topologically closer while dissimilar vectors are repulsed (Figure 4). The NI vectors used for training were profiles isolated from Map Task performances like those analyzed in Figure 2 where uncertainty was verbalized. While the average length of the uncertainty vectors was 96 s, we only used the first 70 s of each for training, as the goal was to develop models of the onset of uncertainty. Projecting into the future from this period might provide estimates of the magnitude and duration of uncertainty, which could be used for providing intelligent feedback during training.

OPEN IN VIEWER

Figure 4.

Topology of the self-organizing map. This figure plots the first 70 s of the 55 NI exemplars used for training the self-organizing map. The wider line indicates the state average. The numbers below each exemplar indicate the average NI bits for the state. The arrows indicate that uncertainty decreases from left to right as well as from bottom to top.

The SOM-states in the upper right (#15 & #16) were populated with 8 of the 10 control (i.e., no verbalized uncertainty) exemplars in the training set; the other two control exemplars were in the topologically adjacent SOM-state #14. These exemplars had low NI levels (< 0.02 bits). The SOM-states furthest away on the topology map, SOM-states #1 & 2, were populated with four of the five medical student exemplars in the training set, which were included to provide variety while training the network. These segments had high NI levels (>0.3 bits) with few distinct peaks. This is consistent with novice healthcare teams having higher NI levels than experienced teams.^34,37

The training set also had a limited number of expert healthcare team performances, and these segments were located in the central region of the topology map at SOM-states # 3, 6, 9 and 10, and showed moderate NI levels (>0.15 bits) with an upslope rather than a peak tendency. While the training set contained a limited number of exemplars (n = 55), they were sufficient to create a topological map that differentiated states of little and high uncertainty, with dynamical variants clustered in between these poles.

These SOM-state designations provide a descriptive context for determining the temporal dynamics of different categories of uncertainty expressed by the submarine navigation team.

The sequential dynamics of the shifting SOM-states are shown for the QM who performed two navigation simulations, the first of which was shown in Figure 3. The dynamics are shown for the 11 Hz frequency of the F4 and Cz sensors to estimate the across-scalp variability. These performances included the Briefings before each Scenario, as well as the Debriefings after the Scenarios (Figure 5).

OPEN IN VIEWER

Figure 5.

Temporal dynamics of the SOM-states. The SOM-state for each second is plotted for the F4 (top) and the Cz (bottom) sensors for the QM. The text labels indicate the training segments, and the bracket in Scenario 1 indicate the periods of Rounds activity.

The expression of the SOM-states differed qualitatively for the Briefing, Scenario, and Debriefing segments, with the Debriefings characterized by shorter durations and a greater diversity of SOM-state expressions.

During the first scenario (Scenario 1) at the Cz sensor there was a repeating motif consisting of the sequential expression of SOM states 8 → 7 → 11 → 12 → 14. This sequence, which aligned with the Rounds sequence, indicates rising uncertainty which peaked, declined, and then returned to near baseline. The F4 sensor showed similar dynamics but without the involvement of SOM-states 7 and 8. During Scenario 2 the lower uncertainty SOM-states 14, 11, and 12 dominated at both sensor locations.

There was task specificity to the SOM state expressions as during the Debriefing segments: the state profiles became more heterogeneous, shorter in duration, and with little expression of the SOM states seen during the Scenario portions. The apparent task specificity with regard to training segment differences suggests that the SOM trained with pre-college student performances on map navigation tasks generalized well to the US Navy submarine navigation team performances.

3.4. Dynamic differences between on- and off-diagonal SOM-state transitions

One of the hallmarks of human behavior is that of all the physiologic, mental, and spatial states we can be in, we often occupy and return to specific states ³⁸ : We wake up, have breakfast and travel to work at the same time on weekdays, using the same transportation, and don’t do so at weekends. Such repetition of behavioral states is particularly likely for teams performing tasks with repeating subtasks, like taking Rounds.

Transition maps were used to identify persistent states of neurodynamic organization, and the transitions among them, for the submarine navigation team. Rather than restrict the analysis to a single team member as in Figure 5, the SOM state transition matrix studies were developed from scalp-averaged NI (i.e., across all frequencies and sensors) for the four team members (Figure 6).

OPEN IN VIEWER

Figure 6.

SOM-state durations and transitions. (a) Histogram of the durations of all SOM-states. (b) Average duration of each SOM-state is shown in black, the standard deviations in gray. (c) Transition matrix From a SOM-state (x axis) To the next state (y axis). (d) The minor state transitions were visualized by first removing the identity-line SOM-states. The color bar shows the probability of transiting to the next state. The identity line separates the transitions that will result in lower uncertainty (above the line) or higher uncertainty (below the line).

The average durations between state changes for the four team members was 31.6 ± 40 s (Figure 6(a)). This duration between state changes for the scalp-wide NI dynamics was similar to that calculated for the 11 Hz frequency using the Matlab®findpeaks.m function (26.3 ± 14 s) or by averaging the durations between SOM state changes in Figure 5, which was 25.4 ± 16 s. The durations at each SOM-state were variable both in terms of the average times (Figure 6(b), black bars) and standard deviations (Figure 6(b), grey bars) with SOM states 1, 5, and 10 dominating. The identity line in the transition matrix indicates that most SOM states were temporally persistent (Figure 6(c)), i.e., these are the SOM states of uncertainty that the team members preferred to return to. There were also 510 SOM-state transitions that the team used to move along the identity line—i.e., From state 6 To state 3 (Figure 6(d)).

The transition probabilities of the off-identity transitions involved in moving from state to state along the diagonal are shown in Figure 6(d). These were not evenly distributed across the state space but tended to divide themselves into those above and below the identity line. With the topology shown in Figure 5, those above the line represented transitions to a state of lower uncertainty. An example would be SOM state 7 where most of the likely transitions lead to a state of lower uncertainty: SOM states 8, 10, &13. Those below the identity line represent transitions to a higher state of uncertainty—an example would be state 5 or 10.

3.5. Can SOM-state transition probabilities forecast future states of uncertainty?

The transition probabilities may provide an indication of the future state of uncertainty for team members or collectively for the team. This was tested using a three minute segment at the end of the scenario when the situation rapidly deteriorated from normal operation to a submarine grounding. The triad monitored was the NV, QM, and the OD. The OD was on the bridge and separated from the navigation team in the control room.

The last 190 s of the simulation are shown (1750 s–1937 s), while the submarine was approaching the harbor entrance. Initially the ship was right of track; then 63 s later it became extremely right of track leading up to the grounding at 180 s (Figure 7).

OPEN IN VIEWER

Figure 7.

Predicting the future uncertainty of the team during an incident. (a) The SOM-state profiles are plotted for the QM, NV, and OD. (b) The predicted SOM-states using the transition probabilities in Figure 6(d). (c) The SOM-state, EEG-PV, and NI profiles for the OD during the grounding incident.

The SOM-state profile is shown for the QM, the NV, and the OD for the 197 s segment (Figure 7(a)) and below each is the predicted SOM-state profile (Figure 7(b)). The predicted SOM-state profile was calculated by assuming the actual starting state of each person and that they would remain in that state until the time of the next transition. At the next transition, the most likely transition from one SOM-state (x axis) to the next SOM-state (y axis) was calculated, which then persisted to the next transition, and so on.

The actual state profile of the QM was dominated by SOM-state #14, which represents little uncertainty. In the predicted state profile, SOM-state #14 also dominated until the end using the matrix probabilities. The actual SOM-state profile of the NV was lower (state #5) than for the QM, indicating increased uncertainty, which was understandable as the boat was right of track. His SOM-state decreased further as a grounding became likely.

The OD had the most complicated SOM-state profile, starting with SOM states 10-11 before decreasing. This was followed by a rapid series of state changes before stabilizing with SOM-state 6, indicating a high level of uncertainty. The predicted SOM-states for the QM and NV were very similar, indicating accurate forecasting of the next state. The prediction was less accurate for the OD, particularly in the middle of the segment when the transitions were occurring in rapid succession.

The detailed dynamics of the OD’s EEG-PV and NI occurring during this segment were plotted to better understand the decreased predictability (Figure 7(c)). The EEG-PV was below average at the beginning of the segment (~0.5 compared with the 1.0 average) and decreased further until ~60 s before the grounding when it rapidly began to rise. The NI levels of the OD began rising shortly after the start of the segment and continued to rise until the grounding when they rapidly dropped off. This continual NI increase is an example of persistent periods of both deactivated and activated EEG power contributing to an in increase in NI.