Learning Driver Models for Automated Vehicles via Knowledge Sharing and Personalization

Abstract

This paper describes a learning framework for driver models of automated vehicles (AVs) via knowledge sharing and personalization. As a result of the inherent variability in the transportation system, exposing AVs to all potential driving scenarios through empirical testing is challenging. This limitation can render AVs unaware of certain critical situations, affecting their safe and efficient operation. To address this challenge, this paper proposes a collaborative training approach that enables the sharing of knowledge among AVs while maintaining personalized models tailored to each vehicle’s unique conditions. The method adopts a federated learning strategy to facilitate privacy-aware collaboration, sharing only model parameters and never raw data. Through experimental simulations, we demonstrate the framework’s capability to (i) learn unseen driving scenarios and (ii) personalize driver models to different driving styles (aggressive and passive). Codes and data are made available at the project page https://github.com/wissamkontar.

Keywords

operations automated/autonomous vehicles car-following traffic flow personalization driver model

The curse of variability stands as a critical barrier in the development and deployment of automated vehicles (AVs) in the open world. Variability in the open world stems from the innate dynamic, stochastic, and unpredictable nature of the transportation system. The driving task can change significantly depending on the traffic state (congestion, free flow, etc.), weather conditions (foggy, snowing, etc.), and roadway design (divided highway, one-way, etc.). It even depends on what other agents (pedestrians, bikers, busses, etc.) are present. Human drivers adapt and respond to these variations instinctively, but replicating such situational knowledge for an AV to maneuver safely in any given scenario, is challenging in light of limited data availability and the difficulty of real-world experimentation. In a recent release of a large-scale AV dataset from Waymo ( 1 ), we see limited variability in the testing environment. For instance, the Waymo open dataset shows the majority of trip logs in sunny (99.3%), daytime (80.6%), and urban scenarios ( 2 ). Another large dataset from nuScenes includes more diverse driving environments and scenarios as some driving logs come from different cities (Boston and Singapore) and locations (urban, residential, and industrial), and does include some rainy and cloudy weather conditions. However, most open-world experiments and data collection (e.g., Cruise, Lyft, Aurora) are being done in dedicated routes and locations with limited exposure to variability in the transportation system. Dedicated experiments in similar driving scenarios allow for a deeper understanding of the AV performance in certain scenarios. However, a breadth of exposure is critically needed to train on extensive scenarios and encounters.

The constraint on data availability is compounded by the difficulty of conducting real-world experiments. The development and testing of AVs require extensive resources and come with inherent safety concerns. Consequently, we see many experiments with exposure to limited scenarios, likely leaving AVs blind to a wide array of encounters. It is thus critical to develop methods to share knowledge across AVs to increase exposure to a wide range of scenarios occurring in the open world.

Recent literature has also uncovered how AVs can exhibit different behaviors on the road based on the underlying design and control logic of these vehicles. For instance, we show in references ( 3 – 6 ), how the car-following (CF) behavior of an AV exhibits a range of behavior depending on the underlying control parameters on spacing, desired speed, and acceleration constraints. Thus, data from AVs—specifically multi-class AVs (i.e., those designed with different performances in mind)—are extremely heterogeneous. This creates another challenge in data-driven learning of driver models, as one needs to create a tailored model for each vehicle that considers its own unique characteristics or desired performance. This process is referred to as personalization hereafter. This study aims to develop a framework through which different AVs can share knowledge from their encounters, yet retain a personalized model tailored to their conditions and unique properties. A driver model here refers to a model capable of safely and effectively maneuvering the AV in various driving scenarios. This driver model controls the vehicle’s perception, decision-making (on acceleration, braking, and steering), and navigation. However, the main scope of this work is in presenting how the sharing of information (through parameter transfer) is done while retaining a personalized model, but not on designing an optimal driver model.

Related Work

With the advent increase of computational power and the sheer amount of data collected in today’s systems, federated learning became a powerful tool with the intent of processing the data where they were created on the edge. The edge here refers to a single device, vehicle, or the like. As a consequence, traditional Internet of Things (IoT) applications have shifted to a decentralized approach termed the Internet of Federated Things (IoFT) ( 7 ). This brought about multiple analytical and computational tools that define how devices are set to collaborate with each other and share information. One of the early forefront tools in federated data analytics is federated averaging (FedAvg), which was tailored for deep learning applications ( 8 ). The idea here is simple; devices in a network structure would collaborate to learn a global deep learning model with the coordination of the central sever. Local devices perform iterations of stochastic gradient descent (SGD) using their data to obtain local parameters of their deep learning model and send those parameters to a central server. Then the central server takes an average of those parameters to update the global model. Since then, several works have refined federated data analytics and tailored it to different applications. Notably, ( 9 , 10 ) scale the application of federated learning into GPs, and general linear models. ( 11 ) presents Fed-ensemble, bringing ensemble methods to federated learning, improving generalization and uncertainty quantification. ( 12 ) tailors a federated algorithm to learn unique and shared features for principal component analysis. And several other models exist, which are reviewed in references ( 7 , 13 , 14 ).

While the application of federated learning in transportation systems has seen some momentum, it has yet to expand. Recently, ( 15 ) presented a survey review of federated learning for connected and AVs. They note several applications of federated learning for in-vehicle human monitoring, steering wheel prediction, object detection, and motion control. Most relevant to this work are the ones related to vehicle trajectory prediction. For example, ( 16 ) uses an encrypted federated network algorithm to learn driver behavior and predict trajectories. ( 17 ) designs a federated deep reinforcement learning for trajectory planning. Most notably, a study by ( 18 ) explored personalization of adaptive cruise controllers via a Gaussian process (GP) model. However, it remains that only a few efforts have been invested with limited scope, and the application domain of AV learning via federated learning remains largely unexplored.

The focus of this paper is different from the available work and explores the potential of federated learning in a different direction. Specifically, we aim to share knowledge across vehicles to learn driving scenarios that might have been missed by individual vehicles because of the variability in the transportation system, while retaining personalization for each vehicle.

Accordingly, this work is motivated by the need to (i) share driving knowledge between different vehicles to increase the exposure of an AV to different driving scenarios/environments, (ii) retain a personalized model for each vehicle under heterogeneous behavior, and (iii) bypass the need to access the raw data for training driving models.

The rest of the paper is organized as follows: The motivation and contribution section describes the problem statement addressed in this work and summarizes major contributions. Then, the methodology section discusses the problem setting and model formulation. In the simulation analysis section, we showcase our method’s capabilities in knowledge sharing and personalization. Finally, the discussion and conclusion section finishes with remarks, limitations, and future directions.

Motivation and Contributions

Motivation

One can question the need to share knowledge through collaboration between vehicles as opposed to just pooling the data and learning one model. We argue that learning one-size-fits-all can lead to misleading results and dilute the distinction between (i) different experienced driving scenarios and (ii) heterogeneous driving behavior from subject vehicles.

For instance, consider the basic unit of a driver model—the CF model—whereby an AV uses its sensor data to regulate its own acceleration/speed such that it follows its leader in a safe manner. The design of such a CF controller for an AV requires specific consideration of desired spacing, speed preference, and comfortable acceleration. We can see such a model in play when we consider the widely adopted linear controller in adaptive cruise control (ACC) and even full self driving systems ( 3 , 19 , 20 ). In a linear controller, the desired spacing is first modeled based on the constant time gap policy as

d_{v}^{*} (t) = v_{v} (t) \times τ_{v}^{*} + δ_{v}^{*}

(1)

where $d_{v}^{*} (t)$ is the desired equilibrium spacing of vehicle $v$ at any time $t$ ; $v_{v} (t)$ is the respective speed of vehicle v; $τ_{v}^{*}$ is the constant time gap; and $δ_{v}^{*}$ is the standstill distance. Accordingly, the deviation from the equilibrium spacing can be written as $Δ d_{v} (t) = d_{v} (t) - d_{v}^{*} (t)$ , where $d_{v} (t)$ represents the actual spacing between vehicle $v$ and its leader $(v - 1)$ at time $t$ , and the speed difference between vehicle $v$ and its leader $(v - 1)$ is $Δ v_{v} (t) = v_{v - 1} (t) - v_{v} (t)$ .

As such, a system state for the AV controller can be described by $x_{v} (t) = [Δ d_{v} (t), Δ v_{v} (t), a_{v} (t)]^{T}$ and input state as $u_{v} (t)$

\begin{matrix} u_{v} (t) = K_{v}^{T} x_{v} (t) \\ K_{v}^{T} = [k_{sv}, k_{vv}, k_{av}] \end{matrix}

(2)

where $k_{sv}$ , $k_{vv}$ , $k_{av}$ are the feedback gains for the deviation from equilibrium spacing $(Δ d_{v} (t))$ , speed difference $(Δ v_{v} (t))$ and acceleration $(a_{v} (t))$ , respectively. The parameter setting of $K_{v}^{T}$ denotes the regulation magnitude for each component in the system state $x_{v} (t)$ , and thus regulates the AV behavior. We refer readers to ( 3 ) for an in-depth analysis on this phenomenon.

It follows that one can design an AV with specific consideration of $d_{v}^{*}$ , and $K_{v}$ in mind. The dichotomy of these parameters is that they are (i) influenced by designer preference (or rider preference as shown in time headway parameter $τ_{v}^{*}$ in commercial ACC), and (ii) influenced by the driving environment. The authors have a prior work that analyzes in depth how an AV can be designed with specific performance in mind, see ( 3 ). However, if data from heterogeneous AVs (i.e., AVs that differ in design and desired performance) are pooled together to train a single driver model, heterogeneity is lost since the training aims to achieve generalization and the underlying assumption is that data are homogeneous. One may work with stochastic models by estimating parameter distributions to regain some heterogeneity in the data. However, personalized models for each AV still cannot be tracked when the data are pooled. Accordingly, personalizing desired speed, headway, and spacing is unenforceable.

It is important to note that in some cases data pooling is not even achievable. Commercial AV data can be protected by propriety rights and privacy concerns. Thus accessing raw data for training purposes may not be available.

We follow the motivation discussed here with a simulation study (presented in the section “Experiment 2: Knowledge Sharing and Personalization Under Heterogeneity”) that shows how pooling data are not ideal under heterogeneity and how our approach can tackle this problem.

Main Contributions

Modeling: In our model, we acknowledge that data coming from each vehicle is uniquely heterogeneous given the variability of driving scenarios, and personal preferences, and thus learning a one-size-fits-all model is not ideal. Instead, our model is personalized to encode unique encounter data and allows for transferring knowledge of unseen driving scenarios from one vehicle to another.

Algorithm: We showcase a training algorithm based on federated learning, where vehicles only need to share iterations of personalized model parameters thus preserving privacy and minimizing communication costs.

Application: We present an application of such framework on learning traffic oscillations by knowledge transfer between three vehicles, and another application on training personalized models under heterogeneity.

Methodology

Problem Setting

Consider the problem setting illustrated in Figure 1. Multiple vehicles are being tested in different driving environments, and each has a different personalized driver model. All of the driving encounters experienced by the vehicles are of interest to us, as the ultimate goal is to design a global driving model capable of maneuvering the vehicle in different driving environments. In perspective, to build such a model, one can extract the data for each vehicle alone and then train a local (i.e., using the vehicle’s own data) driving model. This approach yields a single driving model for each vehicle that is blind to some driving encounters observed by other vehicles (because of different driving environments).

What we seek to accomplish here is to allow vehicles to collaboratively train a global driving model, that allows for (i) knowledge sharing by transfer of information from one vehicle to another, and (ii) a personalized model that focuses on their own local data. The underlying assumption here is that data from each vehicle are uniquely heterogeneous. This heterogeneity is owed to the unique encounter the vehicle is exposed to and its personalized design (e.g., specific desired speed, acceleration constraints, etc.). To achieve this we structure our driving model learning process as shown in Figure 2.

Problem Formulation

Suppose there exist $v \geq 2$ vehicles. Each vehicle $v \in [V] : = {1, . . ., V}$ has local dataset expressed as $D_{v} = {X_{v}, y_{v}}$ with the cardinality of $N_{v}$ . We have an output $y_{v} = [y_{1}, . . ., y_{N}]^{T}$ , and an input $X_{v} = [x_{1}^{T}, . . ., x_{N}^{T}]$ . Here, $X_{v}$ represents the state of the subject vehicle and nearby vehicles, usually a vector of data informed from onboard sensors. For instance, the state data can be relative speed between the subject vehicle and the leading vehicle, spacing, acceleration, jerk, and others. $y_{v}$ represents an action by the AV, as acceleration magnitude for example. Formally, learning a driver model $f_{v}$ given training data $X_{v}$ and $y_{v}$ is a function defined as:

y_{v} (x_{v}) = f_{v} (x_{v}; θ_{v})

(3)

where $f (*)$ can hold any functional form. Most notably, $f (*)$ can be a general deep learning network, GP, reinforcement learning, control model, or a linear model. The main interest here is parameter $θ_{v}$ , which parameterizes $f (*)$ based on training data. The training data are ultimately tied to driving scenarios that vehicle $v$ was exposed to. One can notice that predicting an accurate output $f (x_{v}^{*} : θ_{v})$ ultimately depends on the accurate estimation of $θ_{v}$ . Training $f (*)$ entails minimizing a general loss function, $L (θ_{v}; x_{v}; y_{v})$ . There exist several optimizers to solve the minimization problem of $L$ (e.g., Adams [ 21 ], SGD, etc.). However, the most widely adopted is the SGD approach as it offers generalization power and is extremely efficient ( 22 ). In typical fashion in SGD, the model training is performed in successive iterations. At each iteration of training $t$ , a subset of training data (indexed by $ξ$ ) $X_{v ξ}$ and $y_{v ξ}$ is taken to update model parameters according to the below:

θ_{v}^{(t + 1)} \leftarrow θ_{v}^{(t)} - η^{(t)} g_{v} (θ_{v}^{(t)}; ξ^{(t)})

(4)

where $η^{(t)}$ is the learning rate and $g_{v}$ is the stochastic gradient of the loss function $L$ . The outcome is a driving model $f (x_{v}; θ_{v})$ parameterized by optimal $θ_{v}$ .

Knowledge Sharing through Federated Learning

The above is a general approach to learning a data-driven driver model, based on local data. However, in our approach, we do not want to solely rely on the vehicle $v$ ’s own data that are tied to the driving scenarios it was exposed to, but want to integrate knowledge from other AVs that might have encountered different driving scenarios. As such, we present a collaborative learning scheme that aims at learning a global parameter $\hat{θ}$ that minimizes a global loss function defined as:

L (\hat{θ}) : = \sum_{v = 1}^{V} α_{v} L (θ_{v}; D_{v})

(5)

where $α_{v}$ is the weight parameter for vehicle $v$ in the collaborative training scheme. $α_{v} = \frac{N_{v}}{\sum_{v = 1}^{V} N_{v}}$ , such that $\sum_{v = 1}^{V} α_{v} = 1$ . Consequently, at each communication round, each vehicle runs steps of SGD to estimate its local parameters:

θ_{v}^{(t + 1)} \leftarrow θ_{v}^{(t)} - η^{(t)} g_{v} (θ_{v}^{(t)}; ξ_{v}^{(t)})

(6)

Afterwards, the global coordinator aggregates the model parameters $\hat{θ}$ according to the below rule and sends back $\hat{θ}$ to each vehicle. Essentially, the global coordinator here averages out the local parameters coming out from each individual vehicle’s driver model.

\hat{θ} = \sum_{v = 1}^{V} α_{v} θ_{v}

(7)

In this scheme we are jointly learning a global model—which we consequently use as a prior for personalization in next steps. In this approach, all vehicles participate in the learning structure at each framework, by computing local parameters and sharing with the global coordinator. For instance, in FedAvg ( 8 ) the global coordinator selects clients to perform local computations and update the global model based on local models, and then averages the local models to update the global model. We take a different approach here, but the aggregation step in Equation 7 still follows the FedAvg mechanism given its simplicity and wide adoption. Note that an underlying assumption in our framework is that all vehicles have the same functional form $f (*)$ of the driver model. Thereafter, the global coordinator sends back the updated parameters to each vehicle. As a result, knowledge from each vehicle is shifted around to all collaborating vehicles. We summarize the above in Algorithms 1 and 2.

Algorithm 1: Knowledge Sharing through Federation (Learning a Global Model)
01 Data: number of vehicles $V$ , their initial model parameters $θ$ , number of sharing rounds $s \in S$ , and weight parameters $α_{v}$ 02 for $s = 1 : S$ do 03 Global coordinator broadcasts $θ$ ; 04 for $v \in V$ do 05 $θ_{v}^{(0)} = θ$ ; 06 Perform stochastic gradient descent (SGD) (Algorithm 2) to update local vehicle parameters; 07 end for 08 Aggregation: ${\hat{θ}}_{s} = \sum_{v = 1}^{V} α_{v} θ_{v}$ ; 09 Set $θ = {\hat{θ}}_{s}$ _; 10 end for 11 Return ${\hat{θ}}_{S}$

Algorithm 1: Knowledge Sharing through Federation (Learning a Global Model)

01 Data: number of vehicles

V

, their initial model parameters

θ

, number of sharing rounds

s \in S

, and weight parameters

α_{v}

02 for

s = 1 : S

do
03 Global coordinator broadcasts

θ

;
04 for

v \in V

do
05

θ_{v}^{(0)} = θ

;
06 Perform stochastic gradient descent (SGD)
(Algorithm 2) to update local
vehicle parameters;
07 end for
08 Aggregation:

{\hat{θ}}_{s} = \sum_{v = 1}^{V} α_{v} θ_{v}

;
09 Set

θ = {\hat{θ}}_{s}

_;
10 end for
11 Return

{\hat{θ}}_{S}

Algorithm 2: SGD for Local Vehicle Updates
01 Data: index $v$ , learning rate $η$ , number of local updates $U$ , and vehicle model parameters $θ_{v}^{(0)}$ 02 for $t = 1 : U$ do 03 Perform random sampling to get mini-batch data $D_{v}$ and index it $ξ_{v}^{(t)}$ ; 04 $θ_{v}^{(t + 1)} \leftarrow θ_{v}^{(t)} - η^{(t)} g_{v} (θ_{v}^{(t)}; ξ_{v}^{(t)})$ 05 end for 06 Return $θ_{v}^{(U)}$

Personalization

Consequently, knowledge sharing was achieved by learning a global model parameterized by $\hat{θ}$ and shared with all participating vehicles. It follows that each vehicle needs to personalize $\hat{θ}$ based on its own local data. Through this process, we encode heterogeneous behavior by each vehicle that is described by its own design parameters. Such personalization can be achieved in numerous ways. However, the most commonly used is the regularization concept, where each vehicle uses its own local data $D_{v}$ to minimize a penalized least squares loss function defined as:

mi n_{θ_{personal}} \frac{1}{N_{v}} \sum_{n = 1}^{N_{v}} L (f (x_{v}; θ_{personal}), y_{v}) + ω | | \hat{θ} - θ_{personal} | |_{2}^{2}

(8)

where $ω$ is a positive coefficient, and $\hat{θ}$ is the global parameter learned from Algorithms 1 and 2. This approach personalized $θ_{personal}$ to each vehicle $v \in V$ while retaining global knowledge by encouraging a solution close to $\hat{θ}$ . This regularization encourages $θ_{personal}$ not to drift too far away from $\hat{θ}$ , and it is shown statistically to reduce overfitting and the bias–variance trade-off. Accordingly, each vehicle re-runs Algorithm 2, with $g_{v}$ now based on the regularized loss function defined in Equation 8. Note that one does not necessarily need to do as many iteration steps $(S)$ as done in Algorithms 1 and 2, but a fraction of the steps can suffice.

Simulation Analysis

Simulation Setup

Consider that we have different AVs denoted by $v$ . Here, we adopt a simple driver model setup as the goal is not to analyze the prediction performance of the underlying driver model, nor to design a driver model (several works in the literature exist on that), but rather to see how much knowledge sharing and personalization can be achieved through our proposed training structure. Accordingly, we define our driver model as a CF model that predicts the AV speed (i.e., the output $y_{v}$ ) based on an input of leader speed $(x_{v})$ . We use a $G P$ Regression to design such a driver model. Note that our framework (knowledge sharing and personalization) can work under different functional modes for the driver model, so users can replace the $G P$ with a model of choice. We will further demonstrate later how the $G P$ can be substituted with a neural network (NN) (shown in Figures 9 and 10 and their respective discussion). Consequently, we formulate the driver model as follows:

f_{v} ~ G P (m (\cdot), c (\cdot, \cdot))

(9)

y_{v} = f (x_{v}) + ϵ, ϵ^{i . i . d} ~ N (0, σ_{ε}^{2})

(10)

where $x \in X$ is the input, and $m (\cdot) : X \to R$ is the prior mean function, $c (\cdot, \cdot) : X \times X \to R$ is the prior covariance function, and $ϵ$ is the observational noise with variance $σ_{ε}$ . We further consider the zero mean function, and we assume the covariance function $c (\cdot, \cdot) = σ_{o} k (\cdot, \cdot)$ for a kernel function $k (\cdot, \cdot) : X \times X \to R$ , where $σ_{o}$ is the output variance. We adopt the well-known radial basis function (RBF) as our kernel that is parameterized by the length scale $l$ . Accordingly, we denote $θ_{v} = (σ_{0}, l, σ_{ε})^{T} \in R^{3}$ the parameters to be estimated. To estimate $θ_{v}$ , we use SGD (Algorithm 2) to minimize the scaled negative log marginal likelihood function defined as

L (θ_{v}, X_{v}, y_{v}) = \frac{- 1}{n} logp (y_{v} | X_{v}, θ_{v})

(11)

Accordingly, the collaboration here is based on $θ_{v} = (σ_{0}, l, σ_{ε})^{T} \in R^{3}$ . At every sharing round $S$ (see Algorithm 1), each vehicle performs steps of SGD on its $L (θ_{v}, X_{v}, y_{v})$ to output a local set of parameters $θ_{v}$ (see Algorithm 2). At the aggregation step, a global coordinator then computes $\hat{θ}$ based on the law described in Equation 7, and sets $θ_{v} = \hat{θ}$ . This scheme of collaborative learning and sharing of parameters is what encodes knowledge transfer between participating vehicles. It is important to note that at no point during the collaboration, raw data ( $X_{v}$ and $y_{v}$ ) are shared between vehicles, which makes this approach very computationally efficient and privacy aware. Additionally, the parameters $θ_{v}$ are dependent on the assumed functional form $f (*)$ of the driver model. For instance, if one is using a NN predictor, then $θ_{v}$ would be the weights of the neural net.

Finally, each vehicle re-runs Algorithm 2 based on the regularized loss function described in Equation 8 to compute the final $θ_{personal}$ used in the prediction of AV speed (i.e., predictions from the driver model).

In the same fashion as above, we use NN functional form $f (*)$ for the driver model. The only difference here becomes the set of parameters $θ_{v}$ that are computed by each vehicle and aggregated by the global coordinator. In this case $θ_{V} = (W, b)$ , where $W$ and $b$ are the weights and biases, respectively. However, to offer a different perspective on the driver model, we define our driver model as a CF model that predicts the AV acceleration (i.e., the output $y_{v}$ ) based on an input of leader speed and spacing (between leader and follower).

Experiment 1: Knowledge Sharing to Learn Traffic Oscillations

Now we consider a specific experimental setup for which we want to demonstrate how knowledge is transferred between vehicles. Consider now that we have three AVs $(V = 3)$ . We assume each vehicle was operating under a completely different driving environment. To signify this, we consider three different scenario-based datasets shown in Figure 3: (i) vehicle 1 is operating under constant speed, (ii) vehicle 2 experiences a deceleration maneuver, and (iii) vehicle 3 experiences an acceleration maneuver. Note that these experiments are extracted from the Waymo dataset ( 23 ). We use this dataset to give realism to our experimental setup. Each of the three driving scenarios runs for 19.7 s long run at a 10 Hz resolution. Consequently, each vehicle has a local dataset of $N_{v} = 197$ . In Figure 3 the top row shows the driving scenario for each vehicle. The black curve shows a human driver (leader) oscillation and the orange curve shows the resulting response of AV follower (the Waymo vehicle) (color online only). Recall that our driver model is based on speed input and output. Thus, training our model requires an input, $X_{v}$ , of the leading vehicle (black curve) speed, and predicts an output speed $y_{v}$ of an AV (the orange curve). We note that since our goal here is to test the knowledge sharing framework, we intentionally design such an experiment to (i) restrict the exposure of each vehicle to one single scenario, and (ii) minimize the data to each so that a driver model has limited capability to generalize without the need for collaboration.

The three driving encounters in our setup (Figure 3) represent portions of a full traffic oscillation—constant speed followed by a deceleration and acceleration maneuver to reach a constant speed again. Such oscillations are ubiquitous in traffic systems and are certain to occur in the open world. The goal here is to allow the three vehicles to train a driving model in a collaborative fashion, in such a way that they would individually be. able to respond to a traffic oscillation knowing that none of them had seen such a driving scenario. After collaborative training, we test each vehicle on the full oscillation scenario, seen in Figure 4. The full oscillation in Figure 4, represents an observed empirical oscillation created by a human driven vehicle (HDV)—extracted from the Waymo dataset.

Note on the experimental setup: The setup of this experiment is done in a way to isolate the knowledge, and is purposely designed to reduce complexities associated with large dataset training and different approaches of AV design (e.g., physics-based design). Thus, we only consider a single isolated disturbance (oscillation) here.

Simulation Results

After training the driver model with our approach that focuses on knowledge sharing and personalization, we test against the oscillation shown in Figure 4, whereby the input is the leader speed, and the output is the AV speed. The goal here is to see whether each vehicle (1-2-3) is able to produce an oscillation. Results are shown in Figure 5. Interestingly, we see that “With Knowledge Sharing” (blue curve) each of the vehicles was able to produce a traffic oscillation even though none of the vehicles have full knowledge of an oscillation (recall Figure 3). This is not the case when we look at “Without Knowledge Sharing” (purple curve). However, it is notable that the oscillations “With Knowledge Sharing” are not perfect. This is rather expected given the limited training data, as the focus is not on building a driver model but rather on knowledge sharing. In practice, usually, a driver model would have a complex set of inputs based on the speed of the leader, the position of the leader, and multiple other factors. However, success here is denoted as the ability of each vehicle (1-2-3) to produce an AV response of an oscillation when subjected to one.

The effect of knowledge sharing is specifically amplified when looking at the vehicle 1 profile in Figure 5. Remember that vehicle 1 had only access to its local data which is a constant speed profile. Specifically, it shows that the driver model of vehicle 1 can extract knowledge on deceleration and acceleration phases from vehicles 2 and 3, respectively. This gives it the knowledge to respond to an oscillation. On the contrary, when we train the driver models of vehicles 1-2-3 without knowledge sharing, we notice that they can be blind to some maneuvers (deceleration/acceleration) that inhibit their ability to fully produce an oscillation, see purple curves in Figure 5.

Experiment 2: Knowledge Sharing and Personalization Under Heterogeneity

Now we revisit the problem of data pooling and heterogeneity in AV behavior from the motivation section. We consider here two AVs, each having a different design setup and thus driving behavior. The design of these AVs follows the control formulation presented in Equations 1 and 2. We note that while we purposely exaggerate the AV behavior (aggressive and passive) in our control design to showcase our model’s ability to personalize on drastically different driving styles, the controller remains within the physical range. In fact, we detail in our previous work in Kontar et al. ( 3 ) how real ACC vehicles can exhibit such varying CF behavior. Accordingly, our work in Kontar et al. ( 3 ) informs the choice of parameter setting shown below (aggressive and passive). We refer readers to ( 3 ) for a detailed analysis of that.

Aggressive AV: an AV that is designed to be very responsive to the leader speed and prioritizes the minimization of speed different $Δ v_{v} (t)$ . Specifically, for the aggressive AV we set $K_{v} = [0.01, 10, - 0.01]$ , with $τ_{v}^{*} = 0.5$ , and $δ_{v}^{*} = 5 m$ .

Passive AV: an AV that is designed to be very passive to the leader speed and prioritizes the minimization in the deviation from target spacing $d_{v}^{*} (t)$ . Specifically, we set $K_{v} = [10, 0.01, - 0.01]$ , with $τ_{v}^{*} = 2.5$ , and $δ_{v}^{*} = 7 m$ .

Further, we consider a specific leader (HDV) speed oscillation and then based on the linear controller (Equations 1 and 2) and for the settings described above, we simulate an AV speed profile. This is shown in Figure 6. Note that the leader (HDV) profile is common between the two generated AV profiles. One can directly notice the difference in behavior between the two AVs. The aggressive AV (red) nearly masks its leader (black), while the passive AV (green) is much less reactive. It follows then that the data (i.e., output $y_{v}$ ) generated from the control design of these two AVs exhibit heterogeneous features.

Simulation Results: GP Functional Form and Driver Model with Speed Output and Speed Input

When output data from these two heterogeneous vehicles are pooled together to learn one global driving model, it masks the underlying difference in behavior. This is further illustrated in Figure 7. When we train a driver model based on pooling the speed data from the aggressive and passive AVs, we get a behavior shown in Figure 7 (blue color). The prediction fails to distinguish between aggressive and passive behavior as it tries to achieve generalization.

However, in our proposed learning structure, the personalization step produces $θ_{personal}$ that results in different driver models for each vehicle rather than a one-model-fits-all approach. This allows vehicles to retain their desired behavior while still sharing knowledge. Figure 8 shows the predictions for both aggressive and passive AVs based on our personalized knowledge sharing approach. Table 1 further shows the root mean squared error (RMSE) based on speed for each model. It is evident that under heterogeneity our proposed learning structure can better encode different driving behaviors and thus better fit personalized driver models.

Table 1.

Prediction Error—Gaussian Process Functional Form and Driver Model with Speed Output and Speed Input

RMSE (speed m/s)	Aggressive	Passive
Pooled model	0.53	2.46
Knowledge sharing & personalization	0.12	0.22

Note: RMSE = root mean squared error.

Simulation Results: NN Functional Form and Driver Model with Acceleration Output

To offer a different modeling approach from the above experiment, here we demonstrate our personalization framework with a driver model that uses a NN functional form, and that outputs acceleration prediction (for the AV) based on speed (of leader) and spacing data (between leader and follower). Note that the design of the aggressive and passive vehicles follows the same from the above example in Figure 6

Similarly to the above insights, we see that the knowledge sharing and personalized approach of learning with the new NN functional form and driver model offers better prediction results than the learning based on pooled data that does not consider heterogeneity of behavior. Specifically, Figure 9 shows the prediction performance of the acceleration profile of aggressive and passive vehicles based on a pooled data. Figure 10 shows the prediction performance of the personalization scheme that differentiates between passive and aggressive behavior. We document the prediction errors in Table 2.

Table 2.

Prediction Error—Neural Network (NN) Functional Form and Driver Model with Acceleration Output and Speed and Spacing Inputs

RMSE (acc m/s2)	Aggressive	Passive
Pooled model	0.12	0.25
Knowledge sharing & personalization	0.081	0.074

Note: RMSE = root mean squared error.

Discussion and Conclusion

In this work, we present a way of learning and training driver models for AVs in a collaborative way. Different vehicles can share knowledge with each other through a collaborative iterative process that entails sharing and discovering optimal parameters that minimize a desired global loss function. We also show how vehicles can share knowledge while retaining a personalized model tailored to their own data.

We showcase two experimental applications of the designed model. In the first experiment, three vehicles collaborate to learn a speed oscillation, by decomposing and transferring knowledge between each other. In the second experiment, we show how under heterogeneous AV behavior, learning a driver model while pooling data is not ideal and thus personalization yields better results.

Limitations and Remarks

On the functional form of the driver model: Here we explored $GP$ and $NN$ functional forms. However, in practice, other models might be more widespread such as deep learning or reinforcement learning methods. It is therefore is important to extend to such functional forms. In our framework, we show that the only change would be in the parameters being shared across vehicles when the functional form changes, though the learning framework might need to be tailored for each functional form for optimal results. Additionally, we note that in this work we only considered data-driven functional forms and disregarded physics-based models (or physics-informed data-driven models). This is because of the additional complexity arising from such functional models, especially when it comes to personalization. This is an ongoing research effort by the authors.

On parameter sharing: In our framework, all vehicles share the same driver model ( $GP$ or $NN$ ) with the same feature input and output. However, if one wants to collaboratively share knowledge between vehicles with different functional forms of the driver model (e.g., one vehicle has a NN and another a reinforcement learning model), an extension/modification of the parameter sharing scheme is needed. A way to approach this is to decompose $θ_{s} : = θ_{functional} + θ_{shared}$ , where $θ_{functional}$ are parameters unique to the vehicle driver model, while $θ_{shared}$ come from a globally learned model. Additionally, one can regularize the $θ$ (i.e., assign weights on $θ_{functional}$ and $θ_{shared}$ ) so that a vehicle can put more value on its own driver model and preferences.

On the feature space of the driver model: In this work, we limit the feature space (i.e., input and output of the driver model) to a small space—only considering acceleration, speed, and spacing. We acknowledge that in practice driver models are more complex. However, under large feature spaces, the problem necessitates a prior step of understanding what features in this space are unique to each vehicle and shared across vehicles. Unique features would be used for personalization, while shared ones are used to train the global model. This problem is part of ongoing work by the authors, and we envision a solution rooted in dimension reduction techniques (principal component analysis, autoencoders, and others). In these methods, one can dissect the driver model into a set of low feature spaces used for knowledge sharing and personalization.

Personalization of AVs under performance criteria: Once can directly note that AVs should be designed with strict criteria on performance and behavior (e.g., safety). Thus, when learning under heterogeneous data, it becomes important first to find and filter out those undesired behaviors. For instance, an extreme aggressive AV behavior can be filtered out.

On when to pool data: In some cases, pooling data can be beneficial. For instance, consider the setup in Experiment 1. If vehicles 1-2-3 have similar designs (driver model with same parameter setting), one can pool all the data from different scenarios and train one-modal-that-fits-all. However, pooling data might not even be applicable given that access to some data is restricted (privacy concerns of propriety rights). As such, our approach circumvents this by only sharing parameter values and never raw data. In scenarios where pooling data is the better approach, our framework can serve as a parameter refining step to enhance prediction performance.

We hope that our endeavors in this modeling direction will spur interest and motivate further work on designing complex AV driver models that can safely and efficiently maneuver in the open world. We envision that in practice our work can be used to (i) personalize the behavior of AVs for different preferences (wither at the user-level or traffic-system level), (ii) offer privacy-aware collaboration across vehicles, and (iii) expose an AV to unseen data through collaboration with other vehicles.

Footnotes

Author Contributions

The authors confirm contribution to the paper as follows: study conception and design: Kontar (lead), Zhong, and Ahn; data collection: N/A; analysis and interpretation of results: Kontar (lead), Zhong, and Ahn; draft manuscript preparation: Kontar (lead), Zhong, and Ahn. All authors reviewed the results and approved the final version of the manuscript.

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This research was sponsored by the United States National Science Foundation through Award CMMI 1932932 and the University of Wisconsin-Madison.

ORCID iDs

Wissam Kontar

Xinzhi Zhong

Soyoung Ahn

Data Accessibility Statement

Codes and data used in this paper are made available at the project page .

References

Chen

Qiu

Ai-Rfou

C. R.

Zhou

Yang

, et al. WOMD-LiDAR: Raw Sensor Dataset Benchmark for Motion Forecasting. arXiv Preprint arXiv:2304.03834, 2024.

Zheng

Chen

Sun

Autonomous Vehicle’s Impact on Traffic: Empirical Evidence From Waymo Open Dataset and Implications From Modelling. IEEE Transactions on Intelligent Transportation Systems, Vol. 24, No. 6, pp. 6711–6724.

Kontar

Srivastava

Zhou

Chen

Ahn

On Multi-Class Automated Vehicles: Car-Following Behavior and Its Implications for Traffic Dynamics. Transportation Research Part C: Emerging Technologies, Vol. 128, 2021, p. 103166.

Kontar

Ahn

Bayesian Methods in Automated Vehicle’s Car-Following Uncertainties: Enabling Strategic Decision Making. arXiv Preprint arXiv:2210.13683, 2022.

Zhong

Zhou

Ahn

Chen

Understanding Heterogeneity of Automated Vehicles and Its Traffic-Level Impact: A Stochastic Behavioral Perspective. Transportation Research Part C: Emerging Technologies, Vol. 164, 2024, p. 104667.

Zhong

Zhou

Kamaraj

Zhou

Kontar

Negrut

Lee

J. D.

Ahn

Human-Machine Interaction in Automated Vehicles: Reducing Voluntary Driver Intervention. arXiv Preprint arXiv:2404.05832, 2024.

Kontar

Shi

Yue

Chung

Byon

Chowdhury

Jin

, et al. The Internet of Federated Things (IoFT). IEEE Access, Vol. 9, 2021, pp. 156071–156113.

McMahan

Moore

Ramage

Hampson

Arcas

B. A.

Communication-Efficient Learning of Deep Networks From Decentralized Data. Proc., 20th International Conference on Artificial Intelligence and Statistics, Fort Lauderdale, 2017, pp. 1273–1282.

Yue

Kontar

R. A.

Federated Gaussian Process: Convergence, Automatic Personalization and Multi-Fidelity Modeling. arXiv Preprint arXiv:2111.14008, 2021.

10.

Yue

Kontar

R. A.

Gómez

A. M. E.

Federated Data Analytics: A Study on Linear Models. IISE Transactions, Vol. 56, 2024, pp. 16–28.

11.

Shi

Lai

Al Kontar

Chowdhury

Fed-Ensemble: Ensemble Models in Federated Learning for Improved Generalization and Uncertainty Quantification. arXiv Preprint arXiv:2107.10663, 2021.

12.

Shi

Kontar

R. A.

Personalized PCA: Decoupling Shared and Unique Features. arXiv Preprint arXiv:2207.08041, 2022.

13.

Wang

Charles

Joshi

McMahan

H. B.

Al-Shedivat

Andrew

, et al. A Field Guide to Federated Optimization. arXiv Preprint arXiv:2107.06917, 2021.

14.

Bonawitz

Kairouz

Mcmahan

Ramage

Federated Learning and Privacy. Communications of the ACM, Vol. 65, No. 4, 2022, pp. 90–97.

15.

Chellapandi

V. P.

Yuan

Zak

S. H.

Wang

A Survey of Federated Learning for Connected and Automated Vehicles. arXiv Preprint arXiv:2303.10677, 2023.

16.

Han

Jiang

Federated Learning-Based Trajectory Prediction Model With Privacy Preserving for Intelligent Vehicle. International Journal of Intelligent Systems, Vol. 37, No. 12, 2022, pp. 10861–10879.

17.

Rjoub

Bentahar

Wahab

O. A.

Explainable AI-Based Federated Deep Reinforcement Learning for Trusted Autonomous Driving. 2022 International Wireless Communications and Mobile Computing Conference (IWCMC), Dubrovnik, Croatia, 2022, pp. 318–323. IEEE.

18.

Wang

Han

Tiwari

Work

D. B.

Gaussian Process-Based Personalized Adaptive Cruise Control. IEEE Transactions on Intelligent Transportation Systems, Vol. 23, No. 11, 2022, pp. 21178–21189.

19.

Shladover

S. E.

Nowakowski

X.-Y.

Ferlis

Cooperative Adaptive Cruise Control: Definitions and Operating Concepts. Transportation Research Record: Journal of the Transportation Research Board, 2015. 2489: 145–152.

20.

Milanés

Shladover

S. E.

Modeling Cooperative and Autonomous Adaptive Cruise Control Dynamic Responses Using Experimental Data. Transportation Research Part C: Emerging Technologies, Vol. 48, 2014, pp. 285–300.

21.

Kingma

D. P.

Adam: A Method for Stochastic Optimization. arXiv Preprint arXiv:1412.6980, 2014.

22.

Keskar

N. S.

Mudigere

Nocedal

Smelyanskiy

Tang

P. T. P.

On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima. arXiv Preprint arXiv:1609.04836, 2016.

23.

Zheng

Chen

Zhang

Sun

Processing, Assessing, and Enhancing the Waymo Autonomous Vehicle Open Dataset for Driving Behavior Research. Transportation Research Part C: Emerging Technologies, Vol. 134, 2022, p. 103490.