Robust Constrained Learning-based NMPC enabling reliable mobile robot path tracking

Abstract

This paper presents a Robust Constrained Learning-based Nonlinear Model Predictive Control (RC-LB-NMPC) algorithm for path-tracking in off-road terrain. For mobile robots, constraints may represent solid obstacles or localization limits. As a result, constraint satisfaction is required for safety. Constraint satisfaction is typically guaranteed through the use of accurate, a priori models or robust control. However, accurate models are generally not available for off-road operation. Furthermore, robust controllers are often conservative, since model uncertainty is not updated online. In this work our goal is to use learning to generate low-uncertainty, non-parametric models in situ. Based on these models, the predictive controller computes both linear and angular velocities in real-time, such that the robot drives at or near its capabilities while respecting path and localization constraints. Localization for the controller is provided by an on-board, vision-based mapping and navigation system enabling operation in large-scale, off-road environments. The paper presents experimental results, including over 5 km of travel by a 900 kg skid-steered robot at speeds of up to 2.0 m/s. The result is a robust, learning controller that provides safe, conservative control during initial trials when model uncertainty is high and converges to high-performance, optimal control during later trials when model uncertainty is reduced with experience.

Keywords

Model predictive control Gaussian process robust control

1. Introduction

Model Predictive Control (MPC) is a powerful algorithm capable of computing optimal inputs while seamlessly addressing state and input constraints. At each time-step, the controller solves a finite-horizon optimization problem based on the current state and a model of the system (Rawlings and Mayne, 2009). For mobile robots, state constraints may represent physical obstacles or localization limits, and, as a result, constraint satisfaction is tantamount to safety (Figure 1). However, system models used in practice are often uncertain and constraint satisfaction is difficult to guarantee in general. For example, operation in off-road terrain frequently leads to modeling errors due to surface materials (e.g. snow, sand, grass), terrain topography (e.g. side-slopes, inclines), and complex robot dynamics (Ostafew et al., 2015a).

Fig. 1.

State constraints for mobile robots frequently represent physical obstacles, thus constraint satisfaction is required for safety. We present a Robust Constrained Learning-based Nonlinear MPC algorithm to guarantee constraint satisfaction while improving performance through learning. The algorithm is tested on a 900 kg Clearpath Grizzly traveling up to 2.0 m/s on off-road paths with tight constraints.

Robust Constrained MPC (RC-MPC) is an active area of research and endeavors to provide guarantees on constraint satisfaction when considering uncertain systems (Mayne, 2014). At each time-step, RC-MPC aims to identify inputs such that all plausible predicted trajectories satisfy the given constraints considering a fixed estimate of model uncertainty. However, such approaches are generally conservative since the models are not updated online.

In this paper, we investigate a Robust Constrained Learning-based Nonlinear Model Predictive Control (RC-LB-NMPC) algorithm for a path-repeating mobile robot operating in challenging outdoor terrain. Learning-based algorithms alleviate the need for significant engineering work on identifying and modeling all effects that a model-based controller may be required to mitigate (Nguyen-Tuong and Peters, 2011; Schaal and Atkeson, 2010). Moreover, learning-based algorithms allow for the system to act in anticipation of disturbances.

The algorithm uses a fixed, simple robot model and a learned, nonparametric disturbance model. Disturbances represent measured discrepancies between the a priori model and observed system behavior. Essentially, the algorithm begins with a simple process model and high model uncertainty for the first trial and learns an accurate, low-uncertainty model over successive trials. Disturbances are modeled as a Gaussian process (GP) (Rasmussen, 2006) based on previous experience as a function of state, input, and other relevant variables. We use a Sigma-Point Transform (Julier and Uhlmann, 2004) to efficiently compute the mean and variance of predicted state sequences given the two-component, learned model. We then apply restricted constraints to the predicted sequence such that the 3 $σ$ confidence region is contained within the desired constraints. In this way, the predictive controller automatically computes low-speed, conservative linear and angular inputs during initial trials when model uncertainty is high, and high-performance inputs during later trials when model uncertainty is reduced through learning (Figure 2). Finally, we present extensive experimental results, including over 5 km of travel by a 900 kg skid-steered robot at speeds up to 2.0 m/s.

Fig. 2.

The predictive controller optimizes both the linear and angular velocities over a prediction horizon such that the 3 $σ$ confidence region is contained within the path bounds. (Left) At full speed with an uncertain model, the controller cannot guarantee constraint satisfaction throughout the prediction horizon. (Center) As a result, the controller selects slower, more conservative inputs. (Right) Over successive trials, learning reduces model error and uncertainty, and the controller is able to guarantee constraint satisfaction at the maximum allowed speed.

Localization for the controller is provided by an on-board, Visual Teach & Repeat (VT&R) mapping and navigation algorithm enabling operation in large-scale, off-road environments (Furgale and Barfoot, 2010; Stenning et al., 2013). In the first operational phase, the teach phase, the robot is manually piloted along the desired path. Localization in this phase is obtained relative to the robot’s starting position by Visual Odometry (VO), computing pose changes over sequential images based on extracted feature maps (three-dimensional positions and associated descriptors). At discrete points along the desired path, the algorithm stores the currently viewed feature map. During the repeat phase, the algorithm relocalizes against stored feature maps given the current robot view, generating feedback for a path-tracking controller. As long as a sufficient number of feature matches are made between the live robot view and the stored feature maps, the system generates consistent localization over trials and is able to support a learning control algorithm.

This work differs from traditional constrained NMPC algorithms in two main ways. Firstly, in traditional implementations, the process model is specified a priori and remains unchanged during operation. In this work, we augment the process model with a learned disturbance model in order to predict the mean and uncertainty of effects not captured by the fixed process model. Secondly, constrained NMPC approaches do not typically account for model uncertainty. In this work, we apply robust constraints in real-time considering the learned uncertainty. We provide robust constraint satisfaction when uncertainty is high and increased performance as uncertainty is reduced through learning. This paper is a major extension of previous work. We have previously investigated unconstrained LB-NMPC (Ostafew et al., 2015a) and unconstrained Min–Max LB-NMPC, where control inputs are optimized based on worst-case scenarios (Ostafew et al., 2015b). However, the Min–Max approach does not provide tuning knobs to directly specify tracking error constraints (the only tuning inputs are the MPC weights). Significant additions include robust constraints considering the learned uncertainty and presentation of the necessary nonlinear techniques used to solve the resulting optimization problem. The structure of this paper is as follows. Section 2 relates our work to other research in this field. Sections 3 and 4 describe the proposed RC-LB-NMPC algorithm and implementation details. Section 5 presents experimental results. Finally, Sections 6 and 7 present a discussion and conclusion.

2. Related work

This work is aimed at achieving robust constrained, high-performance path-tracking in spite of unknown disturbances. In the following, we provide a brief review of approaches involving constrained and unconstrained MPC (linear and nonlinear). Then we provide a background on learning control approaches.

2.1. Model Predictive Control

Model Predictive Control (MPC) has received a significant amount of attention in recent years due to its ability to handle complex linear and nonlinear systems with state and input constraints (Mayne, 2014; Rawlings and Mayne, 2009). It has been proposed for mobile robots (Howard et al., 2009; Klančar and Škrjanc, 2007; Krenn et al., 2013; Kühne et al., 2005; Peters and Iagnemma, 2008), unmanned aerial vehicles (Kumar and Michael, 2012; Mueller and D’Andrea, 2013), and humanoid robots (Erez et al., 2013; Gouaillier et al., 2010; Lafaye et al., 2014). However, in each of these examples, the proposed algorithms do not use learned models or account for model uncertainty when applying constraints. As a result, they either require high-accuracy models representing the system and environmental interactions, or risk violating the given constraints.

Tube MPC is a robust constrained algorithm that accounts for model uncertainty when considering constraints (Langson et al., 2004; Mayne et al., 2011). The algorithm applies restricted constraints to nominal predictions such that all possible state sequences based on the given uncertainty satisfy the constraints. Recently, González et al. (2011) and Farrokhsiar et al. (2013) apply Tube MPC to mobile robots. However, unlike their work, our robust constrained algorithm is based on a fixed a priori model and a learned, nonparametric disturbance model. This allows us to apply robust constraints while improving performance through learning. Aswani et al. (2013) propose Learning-based Tube-MPC with a bounded, learned model and use computationally expensive reachability analysis to compute restricted constraints. In this work, state sequences are predicted using an efficient Sigma-Point Transform, and restricted constraints are defined such that the predicted 3 $σ$ confidence region is contained within the true constraints. As a result, our algorithm efficiently computes and applies robust constraints in real time while also improving with experience.

2.2. Learning controllers

Unlike controllers based on fixed models, controllers using learned models gather data over time, incrementally constructing accurate approximations of the true system model. Learned models enable the controller to act in anticipation of disturbances. Learned models are commonly modeled as GPs, Locally Weighted Projection Regression, or Support Vector Regression (Sigaud et al., 2011). In this paper, we model disturbances as a GP based on input–output data from previous trials. This approach enables both model flexibility and consistent uncertainty estimates (Rasmussen, 2006). Modeling a process model as a GP for learning has been previously proposed. For example, Nguyen-Tuong et al. (2008) and Meier et al. (2014) model the inverse dynamics of manipulator arms as a GP based on previous input–output data. Unlike these examples, we model the mean and uncertainty of forward dynamics and propose an RC-NMPC algorithm that seamlessly incorporates the GP model, and robust state and input constraints. Iterative Learning Control (ILC) is another common approach to learning from experience for the control of high-performance robots (Hehn and D’Andrea, 2014; Moore, 2012; Ostafew et al., 2013; Schoellig et al., 2012). ILC constructs an acausal feedforward signal over sequential trials using error information from any previous trial. However, ILC-based learning controllers are only capable of incorporating state and input constraints when constructing the feedforward signal. On the other hand, our model-based RC-LB-NMPC algorithm uses the learned model at each time-step, enabling generalization from learned experiences and real-time, robust constraint satisfaction.

3. Mathematical formulation

At a given sample time, NMPC finds a sequence of control inputs that optimizes the plant behavior over a prediction horizon based on the current state. The first input in the optimal sequence is then applied to the system. The entire process is repeated at the next sample time for the new system state. In this section, we first present our overall robust constrained NMPC algorithm (Figure 3). We then present our methods of predicting uncertain state sequences and estimating disturbances.

Fig. 3.

The RC-LB-NMPC algorithm is composed of two parts: (i) the robust constrained, path-tracking NMPC algorithm based on an a priori process model, and (ii) the GP-based disturbance model. During the first trial, the RC-NMPC algorithm relies solely on the a priori process model to follow the desired path $x_{d}$ , considering the nominal inputs $u_{d}$ . In later trials, the RC-NMPC algorithm uses the disturbance model as a correction to the a priori model at states $a$ , to be defined in Section 3.1. Dashed lines indicate that the signals update the model. In practice, our overall system combines the RC-LB-NMPC algorithm with vision-based localization (Furgale and Barfoot, 2010) for off-road path-tracking.

3.1. Robust Constrained Nonlinear Model Predictive Control

Consider the following nonlinear, state-space system:

x_{k + 1} = f_{t r u e} (x_{k}, u_{k}),

(1)

with observable system state $x_{k} ~ N ({\bar{x}}_{k}, Σ_{k})$ , $x_{k} \in ℝ^{n}$ and control input, $u_{k} \in ℝ^{m}$ , both at time k. The true system, $f_{t rue} (x_{k}, u_{k})$ , is not known exactly and is approximated by the sum of an a priori model and an experience-based, learned model

x_{k + 1} = \overset{a p r i o r i model}{\overset{︷}{f (x_{k}, u_{k})}} + \overset{learned disturbance model}{\overset{︷}{g (a_{k})}}

(2)

with disturbance dependency $a_{k} \in ℝ^{p}$ . The models $f (\cdot)$ and $g (\cdot)$ are nonlinear process models: $f (\cdot)$ is a known process model representing our knowledge of $f_{t rue} (\cdot)$ , and $g (\cdot)$ is an (initially unknown) disturbance model representing discrepancies between the a priori model and the actual system behavior. Disturbances are modeled as a GP (Section 3.3), and thus $g (\cdot)$ is normally distributed $g (\cdot) ~ N (μ (\cdot), Σ_{g p} (\cdot))$ . Previously, we showed that $g (\cdot)$ could be used to learn higher-order dynamics by including historic states in the disturbance dependency (Ostafew et al., 2015a). However, for simplicity, we assume for now that $a_{k} = ({\bar{x}}_{k}, u_{k})$ .

As previously mentioned, the goal of NMPC is to find a set of controls that optimizes the predicted plant behavior over a given horizon. We define the cost function to be minimized over the next K time-steps as

J (\bar{x}, u) = (x_{d} - \bar{x})^{T} Q (x_{d} - \bar{x}) + (u_{d} - u)^{T} R (u_{d} - u),

(3)

where $Q \in ℝ^{Kn \times Kn}$ is positive semi-definite, $R \in ℝ^{Km \times Km}$ is positive definite, $x_{d} = (x_{d, k + 1}, \dots, x_{d, k + K})$ is a sequence of desired states, $x = (x_{k + 1}, \dots, x_{k + K})$ is a sequence of uncertain predicted states, $\bar{x}$ is the sequence of mean values based on $x$ , $u_{d} = (u_{d, k}, \dots, u_{d, k + K - 1})$ is a sequence of desired inputs, and $u = (u_{k}, \dots, u_{k + K - 1})$ is a sequence of inputs. In this work, a sequence of predicted states, $x$ , is iteratively computed using a Sigma-Point Transform (Section 3.2) considering the normally distributed state estimate as provided by the vision-based path localizer, $\hat{x_{k}}$ , a sequence of control inputs, $u$ , and the learned model (2). For mobile robots, desired inputs can be used to schedule travel speed prior to driving according to knowledge of place-specific path characteristics such as roughness or localization reliability.

We also define $2 Kn$ probabilistic state constraints representing upper and lower tracking limits

\begin{matrix} p (e_{m ax, k + i} - e_{k + i} > 0) & > l \\ p (e_{k + i} - e_{m in, k + i} > 0) & > l \end{matrix}} i = 1 \dots K

(4)

where $e_{k + i} = x_{d, k + i} - x_{k + i}$ is the predicted tracking error, $e_{m ax, k + i}$ and $e_{m in, k + i}$ , are upper and lower tracking limits, respectively, $p (A)$ represents the probability of the event A, l is a desired confidence level, and the inequalities are evaluated component-wise. We then compute robust deterministic limits, ${\overset{⌣}{e}}_{m a x, k + i} = e_{m a x, k + i} - α σ_{k + i}$ and ${\overset{⌣}{e}}_{m i n, k + i} = e_{m i n, k + i} - α σ_{k + i}$ , given the predicted uncertainty, $σ_{k + i} = (\sqrt{Σ_{k + i} (1, 1)}, \dots, \sqrt{Σ_{k + i} (n, n)})$ and apply them to the mean predicted states (Figure 4)

{\overset{⌣}{e}}_{m a x, k + i} > {\bar{e}}_{k + i} > {\overset{⌣}{e}}_{m i n, k + i}, i = 1 \dots K

(5)

Fig. 4.

Here we show a 1D example of the computation of robust constraints, ${\overset{⌣}{e}}_{m a x, k}$ and ${\overset{⌣}{e}}_{m i n, k}$ , based on the given constraints, $e_{m ax, k}$ and $e_{m in, k}$ , and the predicted uncertainty sequence, $σ_{k}$ . As the predicted state becomes more uncertain, the constraints are increasingly restricted in order to guarantee that the true state, contained within the 3 $σ$ confidence window, satisfies the given constraint.

where ${\bar{e}}_{k + i} = x_{d, k + i} - {\bar{x}}_{k + i}$ and the inequalities are evaluated component-wise. Typically, $α = 3$ is chosen, representing a confidence level of approximately $0.997$ . In the case that the predicted uncertainty is larger than the given tracking limits, the system is set to stop and request a user input. For example, if the vision-based localization becomes lost, the state uncertainty will likely exceed the given constraints.

In addition, we consider $2 Km$ actuator constraints

u_{m ax, k + i} > u_{k + i} > u_{m in, k + i}, i = 0 \dots K - 1

(6)

where $u_{m ax, k + i}$ and $u_{m in, k + i}$ are defined by the actuator limits and the inequalities are evaluated component-wise. Finally, we define the complete set of constraints, $c_{i} (\bar{x}, u) > 0, i = 1 \dots 2 K (n + m)$ , composed of both robust state (5) and input constraints (6).

Considering the current estimated system state, $\hat{x_{k}}$ , the process model, the constraints, and the cost function, we define the following constrained optimization problem

{x_{o pt}, u_{o pt}} = \arg min_{x, u} J (\bar{x}, u)

(7a)

\begin{matrix} subject to {\bar{x}}_{k + i + 1} = f ({\bar{x}}_{k + i}, u_{k + i}) + g (a_{k + i}), \\ i = 0 \dots K - 1, \end{matrix}

(7b)

c_{i} (\bar{x}, u) > 0, i = 1 \dots n_{c}

(7c)

where the equality constraints are used to enforce the process model, and $n_{c} = 2 K (n + m)$ . We approximate the inequality constraints by introducing a slack variable, $w = (w_{1}, \dots, w_{n_{c}})$ , and a logarithmic barrier term (Boyd and Vandenberghe, 2004)

{x_{o pt}, u_{o pt}, w_{o pt}} = \arg min_{x, u, w} J (\bar{x}, u) - μ \sum_{i = 1}^{n_{c}} \log (w_{i})

(8a)

\begin{matrix} subject to {\bar{x}}_{k + i + 1} = f ({\bar{x}}_{k + i}, u_{k + i}) + g (a_{k + i}) \\ i = 0 \dots K - 1 \end{matrix}

(8b)

c_{i} (\bar{x}, u) - w_{i} = 0, i = 1 \dots n_{c}

(8c)

where $μ$ is a small positive scalar adjusted towards zero throughout the optimization process. Finally, we use Lagrange methods (Boyd and Vandenberghe, 2004) to solve (8) and define the Lagrangian $L (s)$

\begin{matrix} L (s) = J (\bar{x}, u) - μ \sum_{i = 1}^{n_{c}} \log (w_{i}) \\ - \sum_{i = 0}^{K - 1} λ_{i}^{T} ({\bar{x}}_{k + i + 1} - f ({\bar{x}}_{k + i}, u_{k + i}) - g (a_{k + i})) \\ - \sum_{i = 1}^{n_{c}} γ_{i} (c_{i} (\bar{x}, u) - w_{i}) \end{matrix}

(9)

where $λ_{i} \in ℝ^{n}$ , $s = (\bar{x}, u, w, λ, γ)$ , $λ = (λ_{0}, \dots, λ_{K - 1})$ , and $γ = (γ_{1}, \dots, γ_{n_{c}})$ . Considering the necessary condition for optimality, $\nabla L (s) = 0$ , we use Newton’s method and iteratively linearize about an initial guess, $s = \tilde{s} + δ s$

\nabla L (s) \approx \nabla L (\tilde{s}) + \frac{\partial \nabla L (s)}{\partial s} |_{\tilde{s}} δ s

(10)

solve for the value of $δ s$ such that

\nabla L (\tilde{s}) + \frac{\partial \nabla L (s)}{\partial s} |_{\tilde{s}} δ s = 0

(11)

and compute an updated value for $\tilde{s}$

\tilde{s} \leftarrow \tilde{s} + β δ s

(12)

where $0.1 \leq β \leq 1.0$ is selected at each iteration such that $w_{i} > 0, i = 1 \dots n_{c}$ . A good initial guess for $\tilde{s}$ can be drawn from the previous time-step. After iterating to convergence (Algorithm 1), we apply the first element of the resulting optimal control input sequence for one time-step, and start all over at the next time-step.

Algorithm 1: RC-LB-NMPC
Data: $x_{d}$ , $\hat{x_{k}}$ , and $s_{init}$
Result: $x_{opt}$ , $u_{opt}$ , $w_{opt}$
initialization: $\tilde{s} = s_{init}$
while $∥ δ s ∥ > η$ do
Compute state sequence $x$ given the current state
estimate $\hat{x_{k}}$ , $\tilde{s}$ and (2)
Compute robust constraints;
Linearize $\nabla L (s) = 0$ around $\tilde{s}$ and solve for $δ s$
Update $\tilde{s}$ , $\tilde{s} \leftarrow \tilde{s} + β δ s$

3.2. Predicting uncertain trajectories

Since the state $x_{k}$ is normally-distributed and (2) is nonlinear, we use a Sigma-Point Transform (Julier and Uhlmann, 2004) to iteratively predict state sequences (Figure 5) given a sequence of control inputs $u$ and an estimate of the current state from the localizer, $\hat{x_{k}}$ . To predict $x_{k + i + 1}, i = 0 \dots K - 1$ , we define a state $z_{i} = ({\bar{x}}_{k + i}, μ (a_{k + i})) \in ℝ^{2 n}$ representing the mean state and disturbance at time $k + i$ with uncertainty, $P_{i} = d iag (Σ_{k + i}, Σ_{g p} (a_{k + i}))$ . For $i = 0$ , $z_{0}$ is based on the current state estimate, $\hat{x_{k}}$ , otherwise $z_{i}$ is based on a predicted state, $x_{k + i}$ . We compute $4 n + 1$ sigma points, $Z_{i, j} = (X_{i, j}, M_{i, j})$ , where $X_{i, j}$ and $M_{i, j}$ are the sigma points of $x_{k + i}$ and $μ ({\bar{x}}_{k + i}, u_{k + i})$ , respectively

Z_{i, 0} = z_{i}

(13)

Z_{i, j} = z_{i} + \sqrt{2 n + κ} c o l_{j} S_{i}, j = 1 \dots 2 n

(14)

Z_{i, j + 2 n} = z_{i} - \sqrt{2 n + κ} c o l_{j} S_{i}, j = 1 \dots 2 n

(15)

Fig. 5.

Here we show the mean and lateral $3 σ$ boundaries of a predicted sequence $x$ . Since our model uncertainty is normally distributed, we use a Sigma-Point Transform to efficiently compute the mean and uncertainty of predicted state sequences.

where $S_{i} S_{i}^{T} = P_{i}$ with $S_{i} \in ℝ^{2 n \times 2 n}$ derived from the Cholesky decomposition of $P_{i}$ , $c o l_{j} S_{i}$ is the jth column of $S_{i}$ , and $κ$ is a tuning parameter. The sigma points are then passed through the nonlinear model

X_{i + 1, j} = f (X_{i, j}, u_{i}) + M_{i, j}, j = 0 \dots 4 n

(16)

where $f (\cdot)$ is our a priori model. We recombine the sigma points into the predicted mean and uncertainty

{\bar{x}}_{k + i + 1} = \frac{1}{2 n + κ} (κ X_{i + 1, 0} + \frac{1}{2} \sum_{j = 1}^{4 n} X_{i + 1, j})

(17)

\begin{matrix} Σ_{k + i + 1} = \frac{1}{2 n + κ} (κ (X_{i + 1, 0} - {\bar{x}}_{k + i + 1}) (X_{i + 1, 0} - {\bar{x}}_{k + i + 1})^{T} \\ + \frac{1}{2} \sum_{j = 1}^{4 n} (X_{i + 1, j} - {\bar{x}}_{k + i + 1}) (X_{i + 1, j} - {\bar{x}}_{k + i + 1})^{T}) . \end{matrix}

(18)

This process is repeated K times, until the complete sequence $x$ is generated. In this way, the 3 $σ$ confidence region accounts for uncertainty arising from both localization and modeling.

3.3. Gaussian process disturbance model

We model the disturbance $g (\cdot)$ as a GP, which is a function of a disturbance dependency $a$ . Since we provided a detailed explanation of the learned model in previous work (Ostafew et al., 2015a) here we provide only a high-level sketch. The learned model depends on disturbance observations collected during previous trials. At time k, we use the estimated poses $\hat{x_{k}}$ and $\hat{x_{k - 1}}$ from the VT&R system, the disturbance dependency $a_{k - 1}$ and the control input $u_{k - 1}$ to isolate (2) for $\hat{g} (a_{k - 1})$

\hat{g} (a_{k - 1}) = {\bar{x}}_{k} - f ({\bar{x}}_{k - 1}, u_{k - 1}) .

(19)

The resulting data pair, ${a_{k - 1}, \hat{g} (a_{k - 1})}$ , represents an individual experience. For trial j, we collect $N^{(j)}$ data pairs, where $N^{(j)}$ is the number of time-steps it took to travel the length of the path. At the end of each trial, we add newly collected pairs to a large dataset, $D$ , with generally N data pairs, and drop the time-stamp, so that when referring to $a_{D, i}$ or $\hat{g_{D, i}}$ , we mean the ith pair of data in the set $D$ .

In this work, we train a separate GP for each dimension in $g (\cdot) \in ℝ^{n}$ to model disturbances affecting the a priori model. This approach makes the assumption that disturbances are uncorrelated. For simplicity of discussion, we will assume for now that $n = 1$ and denote $\hat{g_{D, i}}$ by ${\hat{g}}_{D, i}$ . The learned model assumes a measured disturbance originates from a GP

\hat{g} (a_{D, i}) ~ G P (0, k (a_{D, i}, a_{D, i}))

(20)

with zero mean and kernel function, $k (a_{D, i}, a_{D, i})$ , to be defined. We assume that each disturbance measurement is corrupted by zero-mean additive noise with variance, $σ_{n}^{2}$ , so that ${\hat{g}}_{D, i} = g_{D, i} + ε, ε ~ N (0, σ_{n}^{2})$ . Then a modeled disturbance, $g (a_{k})$ , and the N observed disturbances, $\hat{g} = ({\hat{g}}_{D, 1}, \dots, {\hat{g}}_{D, N})$ , are jointly Gaussian

[\begin{matrix} \hat{g} \\ g (a_{k}) \end{matrix}] ~ N (0, [\begin{matrix} K & k {(a_{k})}^{T} \\ k (a_{k}) & k (a_{k}, a_{k}) \end{matrix}])

(21)

where $K \in ℝ^{N \times N}$ with $(K)_{i, j} = k (a_{D, i}, a_{D, j})$ , and $k (a_{k}) = [k (a_{k}, a_{D, 1}), k (a_{k}, a_{D, 2}), \dots, k (a_{k}, a_{D, N})]$ . In our case, we use the Squared-Exponential (SE) kernel function (Rasmussen, 2006)

k (a_{i}, a_{j}) = σ_{f}^{2} \exp (- \frac{1}{2} (a_{i} - a_{j})^{T} M^{- 2} (a_{i} - a_{j})) + σ_{n}^{2} δ_{ij}

(22)

where $δ_{ij}$ is the Kronecker delta: that is, $1$ if and only if $i = j$ and $0$ otherwise, and the constants $M$ , $σ_{f}$ , and $σ_{n}$ are hyperparameters. The SE kernel function is an example of a Radial Basis Function (Rasmussen, 2006) and is commonly used to represent continuous functions based on dense data. Further, the SE kernel is continuously and analytically differentiable, enabling the rapid computation of derivatives for (10). In our implementation with $a_{k} \in ℝ^{p}$ , the constant $M$ is a diagonal matrix, $M = diag (m)$ , $m \in ℝ^{p}$ , representing the relevance of each component in $a_{k}$ , while the constants, $σ_{f}^{2}$ and $σ_{n}^{2}$ , represent the process variation and measurement noise, respectively. Finally, we have that the prediction, $g (a_{k})$ , of the disturbance at an arbitrary state, $a_{k}$ , is also Gaussian distributed

g (a_{k}) | \hat{g} ~ N (k (a_{k}) K^{- 1} \hat{g}, k (a_{k}, a_{k}) - k (a_{k}) K^{- 1} k (a_{k})^{T}) .

(23)

Unlike our previous unconstrained LB-NMPC algorithm (Ostafew et al., 2015a), which used only the predicted mean of a disturbance, here we make use of both the predicted mean and variance. As previously mentioned (Section 3.2), the variance of the disturbance is used to predict state sequences with mean and uncertainty given a sequence of control inputs. Then, as detailed in Section 3.1, the predicted uncertainty is used to compute robust constraints in order to provide guarantees on constraint satisfaction. Finally, we include further detail on the storage and retrieval of observations for online operation in Section 4.2.

3.4. Gaussian process hyperparameter selection

Solving for the optimal hyperparameters, $M$ , $σ_{f}^{2}$ , and $σ_{n}^{2}$ , is not currently a real-time process in our experiments. As such, we assume that a suitable set of hyperparameters has been determined prior to each trial based on previous experience (i.e., experience from previous trials). For the first trial, when the robot has no experience, the predicted disturbance is mean zero with relatively high uncertainty. Given a set of experiences, we find the optimal hyperparameters offline by maximizing the log marginal likelihood of collected experiences (Rasmussen, 2006) using a gradient ascent algorithm. In order to avoid local maxima, the algorithm is repeated several times, initialized with different initial values, and the set of hyperparameters resulting in the greatest likelihood is selected.

4. Implementation

4.1. Robot model

In this paper, robots are modeled as unicycle-type vehicles (Figure 6) with state $x_{k} = (x_{k}, y_{k}, θ_{k})$ . At every time-step, the VT&R localization algorithm provides the position of the robot, $\hat{x_{k}}$ , relative to the nearest desired pose by Euclidean distance (Figure 3). As such, we also define the lateral and heading path-tracking errors, $e_{L, k} = y_{d, k} - y_{k}$ and $e_{H, k} = θ_{d, k} - θ_{k}$ , to be used when comparing results. Finally, the robots have two control inputs, their linear and angular velocities, $u_{k} = (v_{c md, k}, ω_{c md, k})$ .

Fig. 6.

Definition of the robot pose variables, $x_{k}$ , $y_{k}$ and $θ_{k}$ , and velocities, $v_{k}$ and $ω_{k}$ . At each time-step the VT&R algorithm provides an estimate of the robot position relative to the nearest desired path pose, $x_{d, j}$ , by Euclidean distance.

When the time between control signal updates is defined as $Δ t$ , the resulting nominal process model employed by the RC-LB-NMPC algorithm is

f (x_{k}, u_{k}) = x_{k} + Δ t [\begin{matrix} \cos θ_{k} & 0 \\ \sin θ_{k} & 0 \\ 0 & 1 \end{matrix}] u_{k}

(24)

which represents a simple kinematic model for our robot; it does not account for dynamics or environmental disturbances. Instead, we depend on the learned model to account for these disturbances. As presented in previous work (Ostafew et al., 2015a), we enable the learned model (2) to represent a system with first-order dynamics by adding historic states and inputs to the disturbance dependency

a_{k} = ({\bar{x}}_{k}, {\bar{v}}_{k - 1}, u_{k}, u_{k - 1})

(25)

where ${\bar{v}}_{k} = ({\bar{v}}_{a ct, k}, {\bar{ω}}_{a ct, k})$ , representing the actual linear and rotational speeds of the robot. These will differ from the commanded ones, $u_{k}$ , owing to the fact that the robots we are working with have underlying control loops that attempt to drive the robot at the commanded velocities. However, the combined dynamics of the robot and these rate controllers are not modeled. We allow the RC-LB-MPC algorithm to learn these dynamics, as well as any other systematic disturbances, based on experience.

In order to build and query the learned model, $g (\cdot)$ , throughout the prediction horizon, we require all of the quantities in (25): $a_{k + i} = ({\bar{x}}_{k + i}, {\bar{v}}_{k - 1 + i}, u_{k + i}, u_{k - 1 + i}), i = 0 \dots K - 1$ . We know $u_{k + i}$ and $u_{k - 1 + i}$ , as these are commanded inputs. We initially obtain the robot position from our vision-based localization system, $x_{k} = \hat{x_{k}}$ , then from the Sigma-Point Transform, (17) and (18). Finally, we compute the velocity state variables, ${\bar{u}}_{k - 1 + i} = ({\bar{v}}_{a ct, k - 1 + i}, {\bar{ω}}_{a ct, k - 1 + i})$ , based on the computed robot positions

{\bar{v}}_{a ct, k - 1 + i} = \frac{\sqrt{{({\bar{x}}_{k + i} - {\bar{x}}_{k - 1 + i})}^{2} + {({\bar{y}}_{k + i} - {\bar{y}}_{k - 1 + i})}^{2}}}{Δ t}

(26)

{\bar{ω}}_{a ct, k - 1 + i} = \frac{({\bar{θ}}_{k + i} - {\bar{θ}}_{k - 1 + i})}{Δ t}

(27)

Since $x_{k}$ and $x_{k - 1}$ come from our vision-based localization system, we are able to initialize the predictive controller with accurate velocity estimates with respect to the ground.

Since obstacle detection in unstructured terrain is still an open problem, lateral and heading tracking error limits are selected prior to experiments and a feasible, desired path is manually defined during the VT&R teach mode considering the given limits. Input constraints are then defined considering the maximum allowed linear and angular speeds, and the curvature of the desired path

v_{m ax, k} = \frac{ω_{l im}}{κ_{k} + ω_{l im} / v_{l im}}

(28)

and $ω_{m ax, k} = ω_{l im}$ , where $κ_{k}$ is the path curvature and $v_{l im}$ and $ω_{l im}$ are the maximum allowed linear and angular speeds. Input constraints are necessary in this work in order to prevent a situation where the algorithm attempts to compensate for a disturbance that physically cannot be addressed (i.e., model discrepancies caused by actuator limits).

4.2. Managing experiences

In order to ensure the RC-LB-NMPC algorithm is executed in constant computation time, our implementation requires the ability to use a subset of the observed experiences when computing a disturbance. In this work, we employ a local model based on a single sliding local model. As experiences are gained, they are stored in bins, $D_{i, l}$ , by path vertex, i, and commanded velocity, $l = ⌊ v_{cmd, k} / v_{bin} ⌋$ , where $v_{bin}$ represents the velocity discretization and $⌊ \cdot ⌋ represents$ the floor function. When the number of experiences in a bin exceeds a threshold, $c_{bin}$ , the oldest experience in the bin is discarded. Then, when computing a control input at the ith vertex, a ‘local’ dataset is created, drawing experiences from bins at nearby path vertices and commanded velocities, $D = {D_{a, b} | a \in {i - c_{vertex}, \dots, i + c_{vertex}}, b \in {l - c_{velocity}, \dots, l + c_{velocity}}}$ , based on thresholds $c_{vertex}$ and $c_{velocity}$ . Thus, models are effectively assembled on demand rather than precomputing hundreds of local models, enabling a constant-time algorithm independent of path length or deployment time.

5. Experimental testing

5.1. Overview

We tested the RC-LB-NMPC algorithm on a 50 kg Clearpath Husky and a 900 kg Clearpath Grizzly in three different experiments with many different surface materials and topographies. This resulted in over 5 km of path-tracking by the RC-LB-NMPC algorithm. The first and second experiments (Sections 5.3 and 5.4) compared unconstrained LB-NMPC, constrained LB-NMPC (C-LB-NMPC), and the proposed RC-LB-NMPC algorithm. The three variants solved the same optimization problem (8) with the following two exceptions. Firstly, state constraints (5) were disabled for the LB-NMPC algorithm. Secondly, the C-LB-NMPC algorithm included only non-robust state constraints (i.e., $α = 0$ when computing (5)). The third experiment (Section 5.5) tested the RC-LB-NMPC algorithm over the course of five trials on an outdoor 900-m-long path with pavement, dirt, sand, grass, inclines, and side slopes (Figure 7). The three experiments demonstrate the presented algorithm’s ability to guarantee constraint satisfaction while improving performance through learning.

Fig. 7.

Testing culminated in the third experiment where the path, shown here, was defined around the University of Toronto Institute for Aerospace Studies (UTIAS) campus. The 900-m-long path passed in between trees and structures and over vegetation, pavement, inclines, and side-slopes. (Imagery: Google)

In all experiments, the controller described in Section 3 was implemented and run in addition to the VT&R software on a Lenovo W530 laptop with an Intel 2.6 Ghz Core i7 processor with 16 GB of RAM. The camera in all tests was a Point Grey Bumblebee XB3 stereo camera. The resulting real-time localization and control signals were generated at approximately 10 Hz. As previously mentioned, hyperparameter selection is currently an offline process, taking up to five minutes for 5,000 accumulated experiences. Since GPS was not available, improvements due to the RC-LB-NMPC algorithm were quantified by the localization provided by the VT&R algorithm. The VT&R algorithm is based on Visual Odometry and provides localization with errors less than 4 cm/m when compared against GPS ground-truth (Stenning et al., 2013).

5.2. Tuning parameters

The performance of the system was adjusted using the NMPC weighting matrices $Q$ and $R$ , the confidence level l and the experience management parameters. The weighting matrices were selected in advance with a 25:1:10 ratio weighting path-tracking errors, angular control inputs, and linear control inputs for the 50 kg Husky and a 25:5:10 ratio for the 900 kg Grizzly robot. The increased weighting on the Grizzly inputs was selected to ensure controller stability at higher speeds. The prediction horizon K was set to $10$ , giving the robot one second to slow down at a reasonable rate when required. The selected confidence level was $l = 0.997$ , resulting in $α = 3$ (i.e., three standard deviations). Local GP models were generated based on a sliding window of size, $c_{vertex} = 5$ and $c_{velocity} = 1$ , where velocities were discretized by $v_{bin} = 0.25$ m/s. The maximum number of experiences per bin, $c_{bin}$ , was set to four, resulting in local models based on up to 180 experiences.

5.3. Experiment 1: Indoor algorithm comparison

The first experiment compared three algorithms (unconstrained LB-NMPC, C-LB-NMPC, and RC-LB-NMPC) over three trials using a 50 kg Clearpath Husky robot on an indoor, flat, concrete surface (Figure 9). As previously mentioned, the algorithms solve the same optimization problem (7) at every time-step, considering identical tuning parameters with the exception of constraints. Since the concrete did not develop tire ruts and the lighting did not change over the course of the experiment, these results provide a clear comparison of the algorithms.

We show the maximum tracking errors and travel times vs. trial in Figure 8. The unconstrained LB-NMPC algorithm results in the largest tracking errors in the first trial, decreasing in later trials through learning. By adding constraints, the C-LB-NMPC algorithm reduced the errors during the first trial, but was overconfident and failed to satisfy the lateral constraints. On the other hand, the RC-LB-NMPC algorithm resulted in constraint satisfaction throughout all trials but incurred increased travel time relative to the other algorithms when the model was uncertain (i.e., during the first trial). Figure 10 shows tracking errors and control inputs vs. distance along the path for all trials. The LB-NMPC and C-LB-NMPC algorithms showed lateral constraint violations at 7, 16, and 24 m along the path (i.e., the turns) in the first trial. The RC-LB-NMPC algorithm avoided such constraint violations in the first trial partly by lowering the speed to approximately 0.5 m/s throughout the path. This speed represents roughly half of the maximum speed possible by the Husky robot. The ability of the RC-LB-NMPC algorithm to optimally compute the linear and angular speeds of the robot in real time while robustly meeting constraints represents one of the advantages of this algorithm. In practice, control algorithms must be capable of reacting robustly to given state constraints in tandem with scheduled speeds. This experiment also shows how the RC-LB-NMPC algorithm produces optimal results similar to the unconstrained algorithm after the learned model has collected experience. In this way, the RC-LB-NMPC algorithm behaves conservatively when the learned model is uncertain, and optimally considering the given cost function when the model uncertainty has been reduced through learning.

Fig. 8.

The maximum tracking errors and travel times vs. trial for the three algorithms in experiment 1. The RC-LB-NMPC algorithm results in constraint satisfaction at the cost of increased travel time (i.e., slower speed) during the first two trials.

Fig. 9.

(Top) The desired path and (bottom) the 50 kg Clearpath Husky at the start of the experiment 1 path. The smooth concrete floor and constant lighting providing identical path conditions for all trials.

Fig. 10.

Tracking errors and commanded inputs vs. distance during the first, second, and third trials of experiment 1. The RC-LB-NMPC algorithm automatically reduces speed in order to meet lateral and heading constraints during the first trial. As model uncertainty is reduced through learning, the RC-LB-NMPC algorithm naturally increases speed and thus performance.

5.4. Experiment 2: Off-road algorithm comparison

The second experiment once again compared the three algorithms (LB-NMPC, C-LB-NMPC, and RC-LB-NMPC). However, in this experiment we tested with a 900 kg Clearpath Grizzly robot on a more challenging and realistic 900-m-long, off-road path (Figure 12) in the University of Toronto Institute for Aerospace Studies (UTIAS) MarsDome. In this experiment, the Grizzly was limited to 2.0 m/s considering path roughness. Since the path was on loose material and the Grizzly is quite heavy, significant ruts developed over the course of the experiment, resulting in evolving disturbances from trial to trial. In this situation, we rely on the learned model uncertainty to capture the effect of the evolving disturbances, and only the RC-LB-NMPC algorithm uses the learned uncertainty to guarantee constraint satisfaction. We show the maximum tracking errors and travel times vs. trial in Figure 11. Without state constraints the LB-NMPC algorithm results in the fastest travel time and also the highest lateral errors of the first trial. During later trials, the errors are reduced through learning, with no notable changes in travel time. By adding constraints, the C-LB-NMPC algorithm reduced the errors during the first trial but still failed to satisfy the lateral constraints. As with experiment 1, the RC-LB-NMPC algorithm resulted in constraint satisfaction throughout all trials and decreased travel time through learning. This confirmed our goal of providing guaranteed constraint satisfaction while improving performance through learning. Figure 13 shows tracking errors and control inputs vs. distance along the path for trials 1, 2, and 10. Without constraints, the LB-NMPC algorithm resulted in the highest speeds and errors. The constraints led the C-LB-NMPC algorithm to reduce speeds in general but the algorithm was overconfident and exceeded limits during each trial. Specifically, the turn at 65m is an example where significant ruts developed, causing wheel slip. Since the RC-LB-NMPC algorithm robustly incorporated learned model uncertainty, the robot successfully passed through this section during each trial. Figure 14 shows measured and predicted disturbances used by the RC-LB-NMPC algorithm. The most complex disturbances affected the angular state of the robot, $g_{(3)}$ .

Fig. 11.

The maximum tracking errors and travel times vs. trial from experiment 2. Of the three algorithms, RC-LB-NMPC is the only one to respect constraints in all trials at the cost of higher travel time.

Fig. 12.

(Top) Lateral constraints outlining the experiment 2 path, along with the RC-LB-NMPC actual path and speed (trial 10). (Bottom) The 900 kg Clearpath Grizzly at the path start.

Fig. 13.

Tracking errors and commanded inputs vs. distance during the first, second, and tenth trials of experiment 2. The most challenging turn occurred at 60m along the path, where significant ruts developed in the loose gravel.

Fig. 14.

Modeled disturbances, $g (\cdot) = (g_{(1)} (\cdot), g_{(2)} (\cdot), g_{(3)} (\cdot))$ , representing disturbances affecting the forward, lateral, and angular velocities of the robot during the first, second, and tenth trials during the first, second, and tenth trials of the second experiment. As the algorithm gathers experience, the model uncertainty decreases and model accuracy increases.

5.5. Experiment 3: Field test

Finally, the third experiment thoroughly tested the RC-LB-NMPC algorithm on a 900-m-long, off-road path repeated five times (Figures 7 and 15). The third path covered a wide range of surfaces, including grass, dirt, pavement, side slopes, and inclines while passing through trees, solid structures, and dense foliage. Once again, we tested with a 900 kg Clearpath Grizzly robot limited to 2.0 m/s considering the path roughness. We show that over the course of the experiment, the RC-LB-NMPC algorithm met constraints and improved performance over sequential trials (Figure 16). Moreover, the experiment shows that the algorithm as formulated was able to consistently deliver control input updates at 10 Hz despite the relatively large dataset gathered over the course of the five trials. Specifically, the learned model had accumulated over 25,000 experiences by the fifth trial, relying on the experience management scheme to extract up to 180 relevant experiences at any given time-step. Figure 17 shows tracking errors and control inputs vs. distance along the path for trials one, two, and five. The first trial is representative of a conservative, non-learning RC-NMPC algorithm: without experience, the predicted disturbance is zero at every time-step but with high uncertainty, capturing the wide range of possible disturbances. After just one trial, the learned model has collected enough experience to significantly reduce uncertainty and increase performance. Modeling disturbances as a GP enables efficient interpolation and extrapolation from data collected during previous trials. Finally, Figure 18 shows tracking error and velocity distributions from trials one and five. The results show that over the course of the experiment, the lateral error distribution widens but continues to fit within the limits. In practice, we do not expect the tracking errors to go to zero, since the RC-LB-NMPC algorithm is an optimal control algorithm, balancing tracking errors and control inputs subject to the constraints.

Fig. 15.

(Center) The experiment 3 desired path roughly overlaid on the UTIAS campus. (Left/Right) images showing path sections with constraint-satisfying terrain (green) and unsafe obstacles (red) highlighted. Without an obstacle detection algorithm, the desired path is manually taught with the required clearance and safe/unsafe areas shown are manually highlighted for illustration purposes. (Satellite Imagery: Google)

Fig. 16.

The maximum tracking errors and travel times vs. trial for experiment 3. Tracking errors met the given constraints while learning reduced the travel time by almost 50%.

Fig. 17.

Tracking errors and commanded inputs vs. distance during the first, second, and fifth trials of experiment 3. Results in the first trial mimic that of a non-learning RC-NMPC algorithm with high model uncertainty representing all possible disturbances and thus conservative inputs.

Fig. 18.

Error and velocity distributions for trials one and five of experiment 3. Learning resulted in a relatively wider distribution of lateral errors as the algorithm became more confident. In practice, the algorithm is seeking to balance tracking errors and control inputs subject to the given contraints. As a result, we do not expect that tracking errors will be driven to zero.

6. Discussion

This work combines concepts of RC-MPC and machine learning in a flexible way. Through extensive experimental results, we have shown practical operation considering a relatively large robot, challenging terrain, and paths requiring tight state constraints. With respect to future work, our modular approach provides the opportunity to investigate and improve both the underlying GP disturbance model and the RC-NMPC algorithm.

With respect to the learned model, Jordan and Mitchell (2015) present a survey of recent trends and prospects for machine learning. Specifically, they highlight the opportunity and need for further research into team-based learning. The goal is to identify effective methods of transferring data between robots such that any improvements found by one robot can be shared by all robots in a team. As opposed to our experiments demonstrating a single robot tracking a single path, one could imagine a large network of paths with many robots improving performance collectively and safely. Also, one of our key requirements for real-time operation is the ability to rapidly identify relevant experiences for our local disturbance model. In our work, experiences are selected based on the nearest path vertex and recent commanded velocity and old experiences are discarded in order to maintain a manageable database. However, further work could involve evaluating the sensitivity of the learned model to this selection process and proposing improvements. For example, we do not currently address the situation where disturbances change predictably over time, such as the arrival of snow or puddles. As such, our algorithm will likely be overconfident driving a snowy path for the first time if the algorithm had previously learned the path disturbances during the summer. Finally, convergence of the learned model is also an open question. In practice, convergence rates are complicated by the evolution of the environment due to the robot’s activity and daily variations. For example, repeating the same path caused ruts to form, which resulted in a change in the disturbances affecting the nominal process model. In such conditions, it is unclear when the algorithm will converge. In general, it is expected that improvements to the disturbance model will result in the largest improvements to overall algorithm performance and reductions in computational complexity.

With respect to the RC-NMPC algorithm, future work includes further integration of the state constraints with a real-time obstacle detection algorithm and investigations into the conservativeness of the algorithm. As previously mentioned, one key benefit of our algorithm is that disturbances and robust constraints are computed in real-time. As such, future work on incorporating new or dynamic obstacles in the optimization problem would be of use for robots exploring new environments or operating in dynamic environments. Secondly, conservativeness of robust algorithms is an ongoing topic for MPC research in general. As can be seen in Figures 10, 13, and 17, the tracking errors remain relatively far from the actual constraints. Reductions in conservativeness will allow for increases in the performance of practical robots.

7. Conclusion

In summary, this paper presents a Robust Constrained Learning-based Nonlinear Model Predictive Control (RC-LB-NMPC) algorithm for a path-repeating, mobile robot operating in challenging off-road terrain. The goal is to guarantee constraint satisfaction while increasing performance through learning. In previous work, we demonstrated unconstrained LB-NMPC, where tracking errors were reduced using real-world experience instead of pre-programing accurate analytical models (Ostafew et al., 2015a). This work represents a major extension where we use the learned model uncertainty to compute and apply robust state and input constraints.

The RC-LB-NMPC algorithm uses a fixed, simple robot model and a learned, nonparametric disturbance model. Disturbances represent measured discrepancies between the a priori model and observed system behavior. We use a Sigma-Point Transform to efficiently compute the mean and variance of predicted state sequences given the two-component, learned, stochastic model. Finally, we apply restricted constraints to the mean predicted sequence such that the 3 $σ$ confidence region is contained within the desired constraints. Localization for the controller is provided by an on-board, vision-based mapping and navigation system enabling operation in large-scale, off-road environments. We provide extensive experimental results comparing unconstrained LB-NMPC, constrained LB-NMPC, and the RC-LB-NMPC algorithm, using two significantly different robots and over 5 km of off-road travel. The results show that the RC-LB-NMPC algorithm efficiently provides real-time robust constraint satisfaction and performance improvements over sequential trials through learning.

Footnotes

Acknowledgements

This research was funded by the Ontario Ministry of Research and Innovation’s Early Researcher Award Program, by the Natural Sciences and Engineering Research Council of Canada (NSERC) through the NSERC Canadian Field Robotics Network (NCFRN), and by Clearpath Robotics.

References

Aswani

Gonzalez

Shankar Sastry

. (2013) Provably safe and robust learning-based model predictive control. Automatica49: 1216–1226.

Boyd

Vandenberghe

(2004) Convex Optimization. Cambridge: Cambridge University Press.

Erez

Lowrey

Tassa

. (2013) An Integrated system for real-time model predictive control of humanoid robots. In: Proceedings of the IEEE-RAS international conference on humanoid robots (humanoids), pp.292–299.

Farrokhsiar

Pavlik

Najjaran

(2013) An integrated robust probing motion planning and control scheme: A tube-based mpc approach. Robotics and Autonomous Systems61(12): 1379–1391.

Furgale

Barfoot

(2010) Visual teach and repeat for long-range rover autonomy. Journal of Field Robotics27(5): 534–560.

González

Fiacchini

Guzmán

. (2011) Robust tube-based predictive control for mobile robots in off-road conditions. Robotics and Autonomous Systems59(10): 711–726.

Gouaillier

Collette

Kilner

(2010) Omni-directional closed-loop walk for NAO. In: Proceedings of the IEEE-RAS international conference on humanoid robots (humanoids), pp.448–454.

Hehn

D’Andrea

(2014) A Frequency domain iterative learning algorithm for high-performance, periodic quadrocopter maneuvers. Mechatronics24(8): 954–965.

Howard

Green

Kelly

(2009) Receding horizon model-predictive control for mobile robot navigation of intricate paths. In: Proceedings of the 7th annual conference on field and service robotics, pp.69–78.

10.

Jordan

Mitchell

(2015) Machine learning: Trends, perspectives, and prospects. Science349(6245): 255–260.

11.

Julier

Uhlmann

(2004) Unscented filtering and nonlinear estimation. Proceedings of the IEEE92(3): 401–422.

12.

Klančar

Škrjanc

(2007) Tracking-error model-based predictive control for mobile robots in real time. Robotics and Autonomous Systems55(6): 460–469.

13.

Krenn

Gibbesch

Binet

. (2013) Model predictive traction and steering control of planetary rovers. In: Proceedings of the 12th symposium on advanced space technologies in robotics and automation, Noordwijk, The Netherlands, pp.1–8.

14.

Kühne

Lages

Silva

JMG

(2005) Mobile robot trajectory tracking using model predictive control. In: Proceedings of the Latin-American robotics symposium, pp.1–7.

15.

Kumar

Michael

(2012) Opportunities and challenges with autonomous micro aerial vehicles. International Journal of Robotics Research31(11): 1279–1291.

16.

Lafaye

Gouaillier

Wieber

(2014) Linear model predictive control of the locomotion of pepper, a humanoid robot with omnidirectional wheels. In: Proceedings of the IEEE-RAS international conference on humanoid robots (humanoids), pp.336–341.

17.

Langson

Chryssochoos

Raković

. (2004) Robust model predictive control using tubes. Automatica40(1): 125–133.

18.

Mayne

(2014) Model predictive control: recent developments and future promise. Automatica50(12): 2967–2986.

19.

Mayne

Kerrigan

Van Wyk

. (2011) Tube-based robust nonlinear model predictive control. International Journal of Robust and Nonlinear Control21(11): 1341–1353.

20.

Meier

Hennig

Schaal

(2014) Efficient bayesian local model learning for control. In: Proceedings of the IEEE international conference on intelligent robotics systems, pp.2244–2249.

21.

Moore

(2012) Iterative Learning Control for Deterministic Systems. Berlin: Springer Science & Business Media.

22.

Mueller

D’Andrea

(2013) A model predictive controller for quadrocopter state interception. In: Proceedings of the IEEE european control conference, pp.1383–1389.

23.

Nguyen-Tuong

Peters

(2011) Model learning for robot control: A survey. Cognitive Processing12(4): 319–340.

24.

Nguyen-Tuong

Peters

Seeger

(2008) Local gaussian process regression for real time online model learning. In: Proceedings of the neural information processing systems conference 21, pp.1193–1200.

25.

Ostafew

Collier

Schoellig

. (2015a) Learning-based nonlinear model predictive control to improve vision-based mobile robot path tracking. Journal of Field Robotics33(1): 133–152.

26.

Ostafew

Schoellig

Barfoot

(2015b) Conservative to confident: treating uncertainty robustly within learning-based control. In: Proceedings of the IEEE conference on robotics and automation, pp.421–427.

27.

Ostafew

Schoellig

Barfoot

(2013) Visual teach and repeat, repeat, repeat: Iterative learning control to improve mobile robot path tracking in challenging outdoor environments. In: Proceedings of the international conference on intelligent robots and systems, pp.176–181.

28.

Peters

Iagnemma

(2008) Mobile robot path tracking of aggressive maneuvers on sloped terrain. In: Proceedings of the international conference on intelligent robots and systems, pp.242–247.

29.

Rasmussen

(2006) Gaussian Processes for Machine Learning. Cambridge: The MIT Press.

30.

Rawlings

Mayne

(2009) Model Predictive Control: Theory and Design. WI, USA: Nob Hill Publishing.

31.

Schaal

Atkeson

(2010) Learning control in robotics. IEEE Robotics & Automation Magazine17(2): 20–29.

32.

Schoellig

Mueller

D’Andrea

(2012) Optimization-based iterative learning for precise quadrocopter trajectory tracking. Autonomous Robots33: 103–127.

33.

Sigaud

Salaün

Padois

(2011) On-line regression algorithms for learning mechanical models of robots: A survey. Robotics and Autonomous Systems59(12): 1115–1129.

34.

Stenning

McManus

Barfoot

(2013) Planning using a network of reusable paths: a physical embodiment of a rapidly exploring random tree. Journal of Field Robotics30(6): 916–950.