Moving Into the Loop: An Investigation of Drivers’ Steering Behavior in Highly Automated Vehicles

Participants

A sample of 64 drivers, between the ages of 25 and 55 and balanced by gender, were recruited from the Madison, WI, area for this experiment. Participants were required to hold a valid driving license for at least 2 years, drive 2,000 miles per year, and be in good health. The experiment lasted approximately 60 min and participants were compensated $30 per hour. This experiment was approved by the Institutional Review Board at the University of Wisconsin–Madison.

Experimental Design and Independent Variables

A 2 × 2 between-subject factorial design combined two automation algorithms and two levels of responsibility. There were 16 participants per condition, balanced by gender. All participants completed one drive.

The two levels of responsibility differed in terms of the instructions to the driver: either the driver or the vehicle was primarily responsible. For both levels of responsibility, lateral and longitudinal control were automated. The two automation algorithms were lane centering and adaptive. The lane centering algorithm precisely tracked the center of the lane, whereas the adaptive algorithm was a combination of lane centering and lane keeping algorithms. The lane keeping algorithm kept the vehicle within the boundaries of the lane, but it wandered back and forth across the lane center. The adaptive algorithm transitioned from lane centering to lane keeping when the automation encountered zones containing latent hazards. The aim of the adaptive control algorithm was to communicate the limits of the automation to the driver. The steering wheel did not provide any feedback to the drivers and hence neither algorithm had any effect on the steering wheel movement or resistance. The effects of the algorithms were only visible through the position of the vehicle on the roadway.

Latent hazards are situations that have the potential to challenge the capacity of the automation, but do not materialize and so the driver does not need to intervene, but should attend to the road (M. Gibson et al., 2016; Siby & Donald, 2015; Vlakveld et al., 2011). In this experiment, drivers encountered four latent hazards (a parked vehicle on the side of the road, construction cones placed in the lane to the right of the driver, parked emergency vehicle on the side of the road, and degraded lane markings).

Apparatus and Scenario

The NADS MiniSim™ was used for this experiment. It consists of a 65-inch screen, placed 54 inches from the driver. It also has a sophisticated gaming steering wheel that provides resistance, but no torque feedback. Participants were instructed to stay in the leftmost lane of a two-lane (12-ft lanes) highway at a constant speed of 55 mph (88.5 kph) when driving manually. Participants could intervene and drive manually at any point in the drive by pressing a button on the dashboard or pressing the brake pedal; however, they could not override the automation by steering.

The scenario began with the participants driving manually for 1 min and then a pre-recorded voice message prompted them to engage the automation. Toward the end of the drive, participants were prompted to switch back to manual control and drove for another minute. The drive included four latent hazards that were placed in four different zones to ensure they did not overlap. Each zone lasted for 5 min and hazards were placed 1 to 4 min into the zone. The adaptive algorithm switched from lane centering to lane keeping 30 s prior to a hazard. This caused the vehicle to drift between the lane boundaries but had no effect on the steering wheel angle. The steering wheel angle remained centered at zero with the two algorithms until the driver began moving it.

Participants were also asked to complete an email sorting task on a tablet, to emulate what drivers might do in an automated vehicle (Lee, 2018). The tablet was placed to the right of the driver and required drivers to look away from the road.

Procedure

Upon arrival, each participant’s drivers’ license was checked. Participants then read and signed the informed consent form. The experimenter then reviewed the capabilities of the vehicle and the driving instructions with the participant. Each participant then completed a 4-min practice to gain familiarity with the simulator. They drove manually for 2 min, then switched to automation for 1 min, and then switched back to manual driving for 1 min. After the practice drive, participants completed a wellness questionnaire to assess symptoms of simulator sickness, and then the capabilities of the vehicle were reviewed again.

In conditions where the driver was primarily responsible, participants were told that they would be driving an automated vehicle where the automation could control both speed and steering, and that they will still be responsible for monitoring the roadway and safe operation of the vehicle and are expected to be available for control at all times and on short notice.

In conditions where automation was primarily responsible, participants were told that they will be driving an automated vehicle that enables them to cede full control of all safety-critical functions under certain traffic or environmental conditions and in those conditions, they could rely heavily on the vehicle to monitor for situations requiring them to take control. They were told that they were expected to be available for occasional control, but with sufficient transition time.

In both conditions, drivers were instructed to switch to manual control if they felt that the vehicle was not able to handle any situation. They were told to turn the automation on by pressing a button and to turn it off by either pressing the button again or pressing on the brake pedal. They were also told that if the automation is engaged, their steering input would have no effect on the vehicle but they did not receive explicit instructions on whether they had to keep their hands on the steering wheel or not.

Data Reduction and Analysis

The data were collected at 60 Hz and reduced to 1 Hz. Dependent variables included standard deviation of the steering wheel angle, standard deviation of the vehicle from the center of the lane, control mode (automated or manual), and proportion of time the eyes were on the road. Transitions between automated and manual control were indicated by the drivers by pressing the automation button—as they were instructed prior to the experiment. To investigate the effect of vicarious steering on takeover performance, we extracted the 15 s before and 15 s after each transition from automated to manual control across the drive. Analyses were done using R 3.5.1 (R Core Team, 2018), the ggplot2 (Wickham, 2016), and dplyr (Wickham, François, Henry, & Müller, 2018) packages.

Steering Behavior Before and After Transitions to Manual Control

Figure 1 summarizes the steering behavior across the entire drive. It shows the mean standard deviation of the steering wheel position for a 1-s rolling window across the drive. The light gray profile in the back represents vicarious steering—steering when the automation was engaged, whereas the darker gray profile represents steering when the automation was disengaged. The gray vertical bars represent the latent hazard zones, which indicate the switch in the algorithm from lane centering to lane keeping in the adaptive conditions. This figure shows the average vicarious steering across all participants throughout the drive. Most of the vicarious steering occurred within the latent hazard zones during the adaptive algorithm conditions.

OPEN IN VIEWER

Figure 1.

Standard deviation of steering wheel angle averaged across participants over the drive. The light gray line represents vicarious steering, and dark gray line represents steering during manual control. The light gray bars represent latent hazard zones.

Focusing on vicarious steering, Figure 2 shows the standard deviation of steering wheel position for each driver (gray circles) and the mean standard deviation of steering wheel position (black dots) across the two levels of responsibility and the two automation algorithms in the periods of engaged automation. The error bars in the figure represent the 95% confidence interval of the means.

OPEN IN VIEWER

Figure 2.

Degree of vicarious steering for levels of instructed responsibility and algorithm type. The mean standard deviation of steering wheel position for a 1-s rolling window for each driver (gray circles) and the mean for all drivers (black points). Error bars represent 95% confidence intervals.

A two-way analysis of variance (ANOVA) compared the effect of instructed responsibility, algorithm, and their interaction on the standard deviation of the steering wheel angle while the automation was engaged. The algorithm significantly increased the steering standard deviation, F(1, 60) = 149.62, p < .001, η_g = 0.71, while there was no significant effect of instructed responsibility and no interaction between responsibility and algorithm, F(1, 60) = 0.18, p = .67, η_g = 0.003 and F(1, 60) = 0.10, p = .75, η_g = 0.002, respectively.

To assess whether vicarious steering indicates increased engagement, we calculated the proportion of time driver’s eyes were on road during the period where automation was engaged, as shown in Figure 3. The results mirrored the effect of vicarious steering. The algorithm increased the proportion of eyes on road input, F(1, 60) = 8.29, p < .001, η_g = 0.12, but there was no significant effect of instructed responsibility and no interaction between responsibility and algorithm, F(1, 60) = 0.01, p = .92, η_g = 0.0002 and F(1, 60) = 0.01, p = .92, η_g = 0.0002, respectively.

OPEN IN VIEWER

Figure 3.

Proportion of drivers’ eyes on road for levels of instructed responsibility and algorithm type. The mean standard deviation of eyes on road proportion for a 1-s rolling window for each driver (gray circles) and the mean for all drivers (black points).

Figure 4 shows the total number of transitions for each participant (gray circles) and the mean number of transitions (black dots) across the entire drive in the four experimental conditions. To disengage automation and transition to manual control, drivers either pushed a button on the dashboard or pressed the brake pedal. A two-way ANOVA compared the effect of the responsibility and algorithm and their interaction on the number of times drivers intervened during the drive. The adaptive control algorithm led to more interventions, F(1, 60) = 22.73, p < .001, η_g = 0.28, but there was no significant effect of the level of responsibility and no interaction between level of responsibility and the algorithm, F(1, 60) = 0.019, p = .89, η_g = 0.0003 and F(1, 60) = 0.368, p = .55, η_g = 0.006, respectively. Even when the two outliers in the adaptive algorithm were removed, the effects persisted.

OPEN IN VIEWER

Figure 4.

Number of transitions from automated to manual driving for levels of instructed responsibility and algorithm type. Total number of transitions for each participant (gray circles) and the mean for all drivers (black points). Error bars represent 95% confidence intervals.

In an additional analysis that considered the non-latent hazard regions exclusively, where both algorithms’ behavior was identical, the effect persisted. More transitions occurred with the adaptive algorithm even when it behaved the same as the lane centering algorithm, F(1, 61) = 4.97, p = .03, η_g = 0.075, with a mean of 2.53 and 1.47 transitions per participant for the adaptive and lane centering algorithms, respectively. An analysis that considered only the latent hazard zone showed a consistent but larger effect of the algorithm, F(1, 253) = 94.13, p < .001, η_g = 0.27, with a mean of 0.6 and 0.06 per participant for the adaptive and lane centering algorithms, respectively.

The Effect of Vicarious Steering on the Transition to Manual Control

To assess the effect of vicarious steering on manual control after a take-over, we considered transitions that lasted for at least 15 s and that followed a minimum of 15 s of automated driving.

Figure 5 shows the steering behavior before and after these transitions; the transition is represented by the gray vertical line at the 16th second. More transitions and more steering activity before the transition occurred with the adaptive algorithm, consistent with the results in Figures 2 and 4. Figure 5 also shows more steering activity after the transition to manual control in those conditions that produced more vicarious steering. The gray area in the background of Figure 5 shows the average percentage of drivers’ eyes on the road, across participants over each second, and indicated that the adaptive algorithm led to a greater proportion of eyes on the road before the transition, which is consistent with vicarious steering indicating increased engagement.

OPEN IN VIEWER

Figure 5.

Standard deviation of the steering wheel angle before and after transitions from automated to manual driving. The transition point is represented by the gray vertical line. The shaded background represents the average percentage of eyes on road across participants.

We examined the effect of vicarious steering during the 5 s prior to a transition on transition quality by considering the vehicle’s standard deviation from the lane center over the 5 s following the transition to manual control (Merat et al., 2014; Zeeb et al., 2016). Figure 6 shows the instantaneous deviation from the center of the lane in the 15 s before and 15 s after the transition to manual control (each line in the figure represents one instance of transition) and suggests that the adaptive algorithm undermined transition quality. The light blue lines in the background represent the lane boundaries. Drivers veered out of the lane more often with the adaptive algorithm.

OPEN IN VIEWER

Figure 6.

Vehicle’s deviation from the center of the lane before and after transition from automated to manual driving (transition represented by the gray line). The blue horizontal lines represent the borders of the 12-ft. lane.

Figure 7 shows the relationship between the standard deviation of steering wheel angle prior to transitions—the amount of vicarious steering—and the standard deviation of the vehicle position from the center of the lane after transitions. Greater vicarious steering was associated with larger lane deviations after the transition. Figure 7 shows a linear model for the relationship between vicarious steering and lane deviation as the light gray line and the black line represent separate linear models for each experimental condition. The differences in the slopes indicate interactions. A full regression model evaluates these interactions. It is important to note the lane width was 12 ft. Table 1 shows that vicarious steering increased the lane deviation from the center of the lane, which indicates lower transition quality, and that when the primary responsibility of the vehicle’s control was assigned to the driver, the deviation was larger. Furthermore, interactions between the responsibility and the automation algorithm and between responsibility and vicarious steering had a smaller effect on increasing the lane deviation after the intervention.

OPEN IN VIEWER

Figure 7.

Effect of vicarious steering on the standard deviation of lane position after transition; the equation on the top represents the regression equation for each condition represented by the black line, whereas the gray line represents the overall regression line of lane deviation on vicarious steering.

OPEN IN VIEWER

Table 1

Summary Statistics for the Full Linear Regression Model of the Transitions Quality (Measured by Standard Deviation of Position From the Center of the Lane) and R ² = 0.332

Variable	Estimate	SE	t-Statistic	p-Value
(Intercept)	0.24	0.11	2.12	.01
Vicarious steering	0.43	0.06	7.10	<.001
Responsibility (driver)	0.53	0.17	3.05	.001
Algorithm (LC)	0.38	0.22	1.75	.1
Vicarious steering × responsibility	−0.22	0.09	−2.59	.05
Vicarious steering × algorithm	−0.24	0.19	−1.25	.21 (ns)
Responsibility × algorithm	−0.80	0.33	−2.43	.05
Vicarious steering × responsibility × algorithm	0.34	0.26	1.39	.17 (ns)

Note. LC = lane centering; ns = not significant.

Effect of Instructed Responsibility and Automation Algorithms on Vicarious Steering

We had expected drivers to have more transitions to manual control and more vicarious steering when the driver was instructed to be primarily responsible for driving because drivers experience the latent hazards shortly after the instructions, but this was not the case. Our results confirm previous research suggesting that drivers are unlikely to follow instructions from owners’ manuals (Novick & Ward, 2006) and that instructions are not an effective remedy for OOTL performance problems (Parasuraman & Manzey, 2010). As with other types of automation, drivers seem to adapt to vehicle automation through direct experience. Drivers in this experiment used the automation immediately after being told about its capabilities and so the conditions we tested are ones that are most likely to show a benefit of instructions. Consequently, the lack of such a benefit is likely to generalize, predicting that owners’ manuals and manufacturers’ admonitions for the driver to remain attentive to partially automated vehicles may have little effect.

In contrast to the instructed responsibility, automation algorithms had a strong effect on vicarious steering and transitions to manual control. Adaptive algorithms, which transitioned from smoothly following the center of the lane to weaving between the lane edges near the latent hazards, prompted more vicarious steering and more interventions compared with the lane centering algorithm. These results confirm a previous study that showed that automation algorithms can increase driver engagement (Price et al., 2017). More generally, consistently high-performing automation leads to complacency and disengagement (Parasuraman, Molloy, & Singh, 1993), and that exposure to automation failures can mitigate this tendency (Bahner, Hüper, & Manzey, 2008). Exposing people to automation failures is a very crude way to convey automation capability, and this study shows a first step toward a more nuanced approach. Future studies might explore how people use behavior to understand the state and intent of others (Blakemore & Decety, 2001). Analysis of drivers interacting with pedestrians shows how the concepts of proxemics and kinetics from communication theory can explain the behavioral communication that supports smooth intersection negations (Domeyer et al., in press). More generally, principles of animation that effectively convey the moods and personalities of animated characters might be applied to reveal the characteristics and capabilities of automation (Lasseter, 1987).

These results show that automation algorithms complement the more traditional use of displays (Seppelt & Lee, 2007, 2019) and that rougher algorithms lead to better calibrated trust and more appropriate reliance (Lee & See, 2004). Drivers were more engaged and more sensitive to hazard in the adaptive algorithm even when the automation behavior was identical to the lane centering algorithm. However, an important caveat to these results is that drivers might not accept algorithms with less smooth performance. Cars are consumer products and drivers might not accept what seems like imperfect technology even though their safety might depend on seeing its imperfections (Ghazizadeh, Lee, & Boyle, 2012).

Vicarious Steering, Engagement, and the Quality of Transitions to Manual Control

This study used vicarious steering as a measure of engagement to assess how well algorithms and instructions can mitigate the OOTL performance problems. From a control theory perspective, vicarious steering occurs when the driver is on-the-loop and engaged through perceptual-motor coupling. Perceptual-motor coupling does not necessarily prevent disengagement, but this study showed promising evidence that greater perceptual-motor coupling, as measured by vicarious steering, led to greater engagement as indicated by more glances to the road and more transitions to manual control.

For SAE Level 2 and 3 automation, vicarious steering might be a useful measure of driver engagement both for system evaluation and as an input to real-time algorithms that assess drivers’ readiness to take over. Our results suggest that vicarious steering might be combined with gaze dispersion (Louw & Merat, 2017; Morando, Victor, & Dozza, 2016) to go beyond simply measuring whether the drivers’ hands are on the wheel and eyes are on the road as indicators of driver engagement. For SAE Level 4 and 5 automation that does not require driver intervention, vicarious steering might be a useful indicator of instances where the algorithm needs to be adjusted to avoid needlessly drawing drivers’ attention to the road.

If greater vicarious steering during periods of latent hazards indicates greater engagement, then one might also expect greater vicarious steering to be associated with better performance after transitioning to manual control. Instead, we found that greater vicarious steering was associated with poorer transitions. There are three explanations for this outcome. First, some vicarious steering might indicate disengagement at the level of attentional lapses. The drivers might have been unaware that automation was engaged and mistakenly thought they were steering when they were not. Second, vicarious steering might reflect greater engagement, but its negative effects might stem from the steering wheel being misaligned with the vehicle trajectory during the transition to manual control (Dinparastdjadid, Lee, Schwarz, Brown, & Gaspar, 2017). The steering wheel in this study provided limited haptic feedback and no effort was made to align the drivers’ steering input with the vehicle trajectory as the vehicle transitioned from automated to manual control. Third, the greater engagement associated with vicarious steering might undermine manual control. It is possible that steering, like other forms of highly skilled behavior, degrades when it receives greater attention (Beilock & Carr, 2001; Engstrom, Johansson, & Ostlund, 2005). Further research is needed to clarify these explanations and the conceptual basis of vicarious steering and how it relates to driver engagement.

As a starting point for this conceptual basis, vicarious steering can be considered as a form of vicarious functioning (Brunswik, 1952). Brunswik defined vicarious functioning in terms of flexibly selecting among multiple possible means to achieve an end. Following such an interpretation, vicarious steering can be thought of as drivers attempting to balance the automation’s control with their own, to produce an outcome more aligned with their expectations. That is, trying to resolve the mismatch between the expected state of the system and its actual state. Arguably, much of human behavior reflects an attempt to resolve such expectations (Engström et al., 2017), and the perceptual-motor coupling that induces vicarious steering might prompt drivers to resolve expectations regarding automation behavior, and carefully tuned feedback through the steering wheel might support active touch (J. J. Gibson, 1962) and further communicate automation capabilities. Automation that enables flexible replacement of one means with another keeps options open and enhances the robustness of control (Degani, Shafto, & Kirlik, 2012). In this way, vicarious steering may be a valuable measure of engagement, and its theoretical grounding might also inspire novel ways to mitigate disengagement that challenges effective human–automation interaction:

An even more insidious problem was a loss of engagement while using the automation. When driving the Tesla in autopilot mode, I found that I was surprisingly slow to react when my car and a recreational vehicle next to me began to come closer together as our lanes on the highway merged. It took extra seconds to realize that the automation was not going to handle the situation. (Endsley, 2017, p. 235)

Limitations

This study used a relatively low-fidelity simulator and participants experienced the automation for a relatively short time, which is not representative of actual driving and so limits the generalization of the results. The short exposure does not provide a good indication of whether drivers would accept automation that does not control in a smooth and consistent manner, nor does it indicate how drivers might adapt to such automation over time. Moreover, when under automated control, the steering wheel did not provide any feedback to the drivers, and the drivers’ input had no effect on the vehicle, which is not representative of most existing automation. With most systems, drivers can override vehicle automation by moving the steering, or pressing the brake or accelerator pedal. Glances to the road in this study were less than the reference model developed by Morando, Victor, and Dozza (2018) based on naturalistic driving in vehicles equipped with Adaptive Cruise Control (ACC) and Lane Keeping Aid (LKA): 86% versus 45%. This may reflect the effect of the automation. Here, we described the automation as capable of controlling the vehicle and encouraged the drivers to engage in a secondary task. Finally, because the steering wheel was not tuned to support smooth transitions, the manual driving performance after takeovers is not likely to be representative of actual driving. Overall, this is a preliminary exploratory study on vicarious steering, and further research is required to investigate vicarious steering and its implications for automation design and evaluation.