Observer-based echo-state neural network control for a class of nonlinear systems

Abstract

The echo-state network is a new structure of recurrent neural networks. Based on the echo-state network, this paper develops an adaptive output feedback control method for a class of perturbed Sngle-Input Single-Output (SISO) nonlinear system in which only the system output is measured. The echo-state network is developed to approximate the control law based on the certainty equivalent approach. A Luenberger like observer is used to estimate the state signals. The echo-state network controller’s parameters are updated on-line using the gradient of descent method. The overall adaptive scheme guarantees that all signals involved are bounded and the output of the closed-loop system will asymptotically track the desired output trajectory without using a supervisory control term. Two nonlinear systems are used to verify the effectiveness of the proposed method.

Keywords

Recurrent neural networks echo state neural networks output feedback control of nonlinear systems

Introduction

Since the seminal paper of Narendra and Parthasarathy (1990) was published, the neural network control schemes have had an active impact on the control community. Neural networks have the merits of massive parallelism, fast adaptability, and accurate approximation ability for any complicated nonlinear system. Thus, neural networks have attracted a lot of research (Ahmed and Faouz, 2015; Chen and Khalil, 1995; Chen and Lin, 1994; Fu et al., 2013; Jianhua et al., 2015; Wenkang et al., 2014; Xie et al., 2009) especially in the area of control and identiﬁcation.

In the adaptive control schemes of nonlinear systems, many schemes based on neural networks have been proposed to obtain control performance (Belarbi and Chemachema, 2007; Ge et al., 1999; Polycarpou, 1996; Sanner and Slotine, 1992; Tang et al., 2006; Wai, 2003; Wang et al., 2014). A key assumption of these methods is that all the states of the plant are available for feedback. In practice, the state feedback control does not always hold because the system states are not measurable. Estimation of states from the system output for the design of a direct adaptive neural controller is required. In Sridar and Khalil (2000), Chien et al. (2011), Castaneda and Esquivel (2012) and Chemachema (2012), the observer-based adaptive neural network controllers are proposed for a certain class of unknown nonlinear system. In the case of multiple input-multiple output nonlinear system, many adaptive output feedback control schemes are proposed (Artemis et al., 2012; Chen et al., 2010; Li et al., 2011). In these schemes, the neural networks are used in the feed-forward topology, i.e. multilayer perceptron (MLP) and the RBF neural networks that suﬀer from a large number of neurons and the weight updates do not use the internal neural network information.

The recurrent neural networks (RNNs) have been developed to address the previous drawbacks (Ku and Lee, 1995; Lee and Teng, 2000; Williams and Zisper, 1990). This can be done by applying dynamic feedback structures, in which the present activation state is a function of the previous activation state and the present inputs (Ozdemir et al., 2011; Townley et al., 2000). RNNs have been introduced in adaptive control schemes (Chun–Fei, 2014; Chun–Fei et al., 2012; El–Sousy, 2013; Li et al., 2014; Michael et al., 2014; Minrui et al., 2006). However, the main problem within RNNs is the computational complexity associated with the update of the weight matrix. The training and analysis of RNNs is not straightforward (Mandic and Chambers, 2001) and suﬀers from a variety of problems, such as the slow training resulting from the computational complexity and the possibility of instability.

The echo state network (ESTN) (Jaeger, 2001) and liquid state network (LSN) (Maass et al., 2002) are modiﬁed paradigms of recurrent neural network (RNN) structures with simpler training methods. Both are motivated by recent neurophysiologic experiments (Jaeger and Haas, 2004). The kernel part of ESTN is a single reservoir with a large number of neurons that are randomly interconnected and/or self-connected. The reservoir itself is ﬁxed, once it is selected. During the training process of ESTN, only the output connections are trained through on line linear regression or on line methods, such as the recursive least square (RLS) or another supervisor training methods (Jaeger and Haas, 2004). Unlike other recurrent architectures, the use of random weights in the ESTN and the training algorithm greatly simpliﬁes the training of the network and eliminates the issues of stability, convergence, and local minima (Mark and Harris, 2007)). Hence, ESTN has been successfully applied for many applications as in the chaotic time series prediction (Jaeger and Haas, 2004; Zhiwei and Min, 2007); in the identiﬁcation of dynamical systems (Jaeger, 2007; Xu et al., 2005); in the online design of a wide-area monitor for a multi-machine power system (Venayagamoorthy, 2007); the ESTN was applied to the identiﬁcation of unknown nonlinear systems and hence a real time predictive control scheme has performed (Pan and Wang, 2012). Seong and Lee (2013) proposed a new adaptive fuzzy wavelet echo state network algorithm to improve performance in terms of approximating unknown uncertainties in conventional neural network algorithms. Moreover, in Seong and Lee (2014), they reported a funnel dynamic surface control combined with fuzzy ESTN for the prescribed tracking performance of a strict feedback multi inputmulti-output (MIMO) nonlinear dynamic system. Guofa et al. (2015) reported an ESTN-based output feedback dynamic surface control, maintaining the prescribed performance of a special class of MIMO nonlinear system. In Suping and Xizhen (2015), a dual adaptive control based on the ESTN for the trajectory tracking of a wheeled mobile robot has been developed.

Motivated by the aforementioned discussion, this paper proposes a direct adaptive control based ESTN with an observer for the control of a certain class of disturbed unknown nonlinear dynamical system, in which only the system output can be measured. Similarly, Mahmoud and Elshenawy (2015) introduced the estimate of the control error to derive the updating laws of the ESTN controller. A key assumption in Mahmoud and Elshenawy (2015) is that all the states are available for measurement. In this work, this assumption is removed and then the state feedback control problem is converted to an output feedback control problem. Moreover, the estimated error signal is introduced to obtain the updating laws of the ESTN controller weights directly using the Back-propagation method. A Luenberger-like observer is used to estimate the state needed for the feedback that avoids the peaking phenomenon in the transient behaviour of the high gain observer as in Ge et al. (1999) and Chien et al. (2011). The convergence of the controller weights is proven. Furthermore, the Lyapunov direct method is employed to prove the global exponential boundedness of all the signals involved in the closed loop, and hence the stability of the system. In contrast to the literature (Castaneda and Esquivel, 2012; Chemachema, 2012; Chien et al., 2011; Sridar and Khalil, 2000), developing the ESTN in the output feedback control scheme leads to visible computations and easier implementation. Moreover, the proposed algorithm does not use any additional supervisory control terms such as in Lin and Peng (2004) and Belarbi and Chemachema (2007) to alleviate the constraints of the known bounds on the nonlinearities. The performance of the proposed scheme is evaluated using two nonlinear systems; nonlinear mass-spring-damper system, and one-link rigid robotic manipulator system.

The rest of the paper is organized as follows. The next section provides the nonlinear system and the problem formulation. Then, the structure and the learning of the ESTN are described. The proposed output feedback control based on the ESTN is presented and two simulation examples are given. Finally, the conclusion of the work is given.

Problem formulation

Consider the nth order nonlinear dynamical system expressed in the affine form

\begin{array}{l} {\overset{\cdot}{x}}_{1} = x_{2} \\ {\overset{\cdot}{x}}_{2} = x_{3} \\ . \\ . \\ . \\ {\overset{\cdot}{x}}_{n} = f (x_{1}, x_{2}, \dots, x_{n}) + g (x_{1}, x_{2}, \dots, x_{n}) u + d \\ y = x_{1} \end{array}

(1)

where $u \in ℝ$ , $y \in R$ are the control input and system output respectively; d is an external bounded disturbance; f and g are nonlinear functions. We can rewrite equation (1) in the following state space form

\begin{matrix} \bar{\overset{\cdot}{x}} = A \bar{x} + B [f (\bar{x}) + g (\bar{x}) u + d] \\ y = C^{T} \bar{x} \end{matrix}

(2)

where

A = [\begin{matrix} 0 & 1 & 0 & . & . & . & 0 \\ 0 & 0 & 1 & . & . & . & 0 \\ . & . & . & . & . & . & . \\ 0 & 0 & 0 & . & . & . & 1 \\ 0 & 0 & 0 & 0 & 0 & 0 & 0 \end{matrix}], B = [\begin{matrix} 0 \\ 0 \\ . \\ . \\ . \\ 0 \\ 1 \end{matrix}], C = [\begin{matrix} 1 \\ 0 \\ . \\ . \\ . \\ 0 \\ 0 \end{matrix}]

$\bar{x} = {[x_{1}, x_{2}, \dots, x_{n}]}^{T} \in ℝ^{n}$ is the state vector. The reference smooth signal ${\bar{y}}_{r}$ and the tracking error $\bar{e}$ can be defined as

\begin{matrix} {\bar{y}}_{r} = [y_{r}, {\overset{\cdot}{y}}_{r}, \dots, y_{r}^{(n - 1)}] \in R^{n} \\ \bar{e} = {\bar{y}}_{r} - \bar{x} = [e, \overset{\cdot}{e}, \dots, e^{(n - 1)}]^{T} \in R^{n} \\ e = y_{r} - x_{1} = y_{r} - y \in R \end{matrix}

If the system (1) is free of external disturbance d, the system states are available to measure, and both $f (\bar{x})$ and $g (\bar{x})$ are known, then a control law based on the certainty equivalent approach can be given by (Chemachema, 2012; Lin and Peng, 2004; Wang, 1994; Wang et al., 2010)

u^{*} = \frac{1}{g (\bar{x})} [- f (\bar{x}) + y_{r}^{(n)} + K_{c}^{T} \bar{e}]

(3)

where $K_{c} = [k_{1 c}, k_{2 c}, \dots, k_{nc}]^{T}$ is the state feedback gain vector, chosen such that all roots of the polynomial $p (s) = s^{n} + k_{nc} s^{n - 1} + \dots + k_{1 c} = 0$ are in the left hand side. Substituting the law given by (3) into system (1) with $d = 0$ , we have

\begin{matrix} e^{(n)} + k_{nc} e^{(n - 1)} + \dots + k_{1 c} e = 0 \end{matrix}

(4)

where the main objective of control is lim $e (t)_{t \to \infty} = 0$ . When the system states are not measured, an observer is needed to estimate the system states to implement the control law (3). Based on the states estimation, the control law (3) can be approximately as follows

\begin{matrix} u^{*} ≅ \frac{1}{g (\bar{\hat{x}})} [- f (\bar{\hat{x}}) + y_{r}^{(n)} + K_{c}^{T} \bar{\hat{e}}] \end{matrix}

(5)

where $\bar{\hat{x}}$ is the estimation state vector, and $\bar{\hat{e}} = {\bar{y}}_{r} - \bar{\hat{x}} = [\hat{e}, \overset{\cdot}{\hat{e}}, \dots, {\hat{e}}^{(n - 1)}]^{T} \in R^{n}$ is the estimation error vector.

In this paper, the following assumption is assumed for the system (1).

Assumption 1. $f (\bar{x})$ and $g (\bar{x})$ are unknown nonlinear and bounded functions, $g (x) \neq 0$ for all values of x, and only the system output y is assumed to be measurable. The control objective is to force the system output y to follow a given bounded smooth reference signal $y_{r}$ , and the derivatives of $y_{r}$ up to $n^{th}$ order exist and are bounded.

According to the above assumption, a control law of the form (5) is not possible, since the nonlinear functions $f (\bar{\hat{x}})$ and $g (\bar{\hat{x}})$ are unknown. In the literature, two distinct approaches have been developed to implement the controller (5), direct and indirect approaches. In the direct approach, an approximation is used to estimate the control law and then the parameters of the estimator are adjusted to meet the required control objective (Chemachema, 2012; Chi–Hsu et al., 2002; Chun–Fei et al., 2012; Lin and Peng, 2004). Unlike the direct approach, the indirect adaptive approach uses an approximation to estimate the system nonlinearities $f (\bar{\hat{x}})$ and $g (\bar{\hat{x}})$ , and then the control law (5) is calculated based on these estimates (Tonga et al., 2004; Wang et al., 2010; Yih–Chuang et al., 1999). The main objective of this article is to develop a direct adaptive ESTN controller that can approximate the ideal control law based on an observer to estimate the system states. In addition, all the signals of the closed loop system must be bounded and the system output must track the reference with a small tracking error.

The echo state neural network

The echo-state network is one of the reservoir computational methods. The basic idea is that input signals are fed into fixed nonlinear dynamical system, called dynamic reservoirs which are composed of randomly connected neurons. Only the output connections, the readout, are trained by simple linear regression. This section describes the structure and the learning of the ESTN controller. We consider ESTN with K input units, N internal network units and one output unit. Let $u (t) = [u_{1} (t), \dots, u_{K} (t)]$ be the K-dimensional input vector at time step t, $υ (t) = [υ_{1} (t), \dots, υ_{N} (t)]$ be the N-dimensional reservoir activation state vector, and $y (t)$ is the network output. The connection weights are $W \in R^{N \times N}$ for the input weights, $W^{in} \in R^{N \times K}$ for the internal connections, $W^{bias} \in R^{N \times 1}$ for bias, and $W^{out} \in R^{N}$ for the connections to the output unit. Figure 1 shows the ESTN architecture considered here.

The state and the output equations of the ESTN are given by the following

υ (t + 1) = f (W υ (t) + W^{in} u (t) + W^{bias})

(6)

y (t) = W^{out} υ (t + 1)

(7)

where $f = [f_{1}, \dots, f_{N}]$ are the internal unit output functions (typically tanh functions). An additional nonlinearity can be applied to $y (t)$ in equation (7), as well as the feedback connection from output $y (t - 1)$ to the internal state $υ (t)$ in equation (6).

The main idea of the ESTN’s learning is that the weights $W$ , $W^{in}$ , and $W^{bias}$ are fixed without adaptation, whereas only the readout weights, $W^{out}$ will be adapted. The existence of the echo state property is critical to ensure adequate operation of the dynamic reservoir of ESTN. This can be performed by the following two steps:

Randomly choose the internal weights $W$ , which is typically drawn from a uniform distribution over a symmetric interval.

Normalize $W$ by dividing it by its largest eigenvalue $λ_{max}$ .

The input and the bias connection weights $W^{in}$ and $W^{bias}$ can be initialized arbitrarily (Jaeger and Haas, 2004). Learning the output weights $W^{out}$ in (7) can be determined by solving a system of linear equations

Y_{target} = W^{out} V

(8)

with respect to $W^{out}$ , where $V \in R^{N \times T}$ are all $V (t)$ produced by presenting the reservoir with $u (t)$ in (6), and $Y_{target} \in R^{L \times T}$ are all the desired outputs of the network $y_{target} (t)$ ; both collected into respective matrices over the training period $t = 1, \dots, T$ . To solve (8), one usually employs a method for finding least squares solutions of the optimization problem

min_{{\hat{W}}^{out}} = ‖ W^{out} V - Y_{target} ‖

(9)

One direct method is to calculate the pseudo-inverse $V^{+}$ of V and the optimum output weights $W^{out}$ can be given as

W^{out} = Y_{target} V^{+}

(10)

Some applications require online adaptation, e.g. in on-line adaptive control schemes (Waegeman et al., 2012). In such cases one typically minimizes an error that is exponentially discounted going back in time. The simplest way to train $W^{out}$ is to use the stochastic gradient descent method. The method is familiar to the least mean squares (LMS) algorithm in linear signal processing (Farhang–Boroujeny, 1998), and has many extensions and modifications. An alternative to LMS, known as the recursive least squares (RLS) algorithm, unlike the LMS it is insensitive to the detrimental effects of eigenvalue spread of $V^{T} V$ and boasts a much faster convergence because it is a second-order method. The classical back propagation algorithm and the back propagation through time are other powerful methods for online training of the ESTN that can be used.

Observer-based echo-state network controller

In this section, our task is to design an ESTN to approximate the controller (3) and develop the tuning rule of the parameters of the ESTN. We consider an ESTN with a single output to estimate the control law (3) such that its reservoir state and output equations are

υ (t) = f (W υ (t - 1) + W^{in} z (t) + W^{bias})

(11)

\hat{u} (t) = W^{out} (t) υ (t)

(12)

where $z (t) = [- {\bar{\hat{x}}}^{T}, y_{r}^{(n)}, K_{c}^{T} \bar{\hat{e}}]^{T}$ . By substituting (12) into (2) and using the estimated states, we can formulate the tracking error equation according to the following form

\begin{matrix} \bar{\overset{\cdot}{e}} = A \bar{e} + B (y_{r}^{(n)} - f (\bar{\hat{x}}) - g (\bar{\hat{x}}) \hat{u} - d) \\ e = C^{T} \bar{e} \end{matrix}

(13)

Adding and subtracting $B K_{c}^{T} \bar{\hat{e}}$ in the right hand side of (13) we have

\begin{matrix} \bar{\overset{\cdot}{e}} = A \bar{e} - B K_{c}^{T} \bar{\hat{e}} + B (y_{r}^{(n)} + K_{c}^{T} \bar{\hat{e}} - f (\bar{\hat{x}}) - g (\bar{\hat{x}}) \hat{u} - d) \\ e = C^{T} \bar{e} \end{matrix}

(14)

Using (5), the term $y_{r}^{(n)} + K_{c}^{T} \bar{\hat{e}} - f (\bar{\hat{x}})$ in the right side of (14) can be replaced by $g (\bar{\hat{x}}) u^{*}$ . Then, the error equation (14) can be rewritten as

\begin{matrix} \bar{\overset{\cdot}{e}} = A \bar{e} - B K_{c}^{T} \bar{\hat{e}} + B (g (\bar{\hat{x}}) (u^{*} - \hat{u}) - d) \\ e = C^{T} \bar{e} \end{matrix}

(15)

Due to the fact that the states of the system can not be measured, the tracking error $\bar{e}$ can not be implemented. Consider the following Luenberger like observer to estimate $\bar{e}$ in (15)

\begin{matrix} \bar{\overset{\cdot}{\hat{e}}} = A \bar{\hat{e}} - B K_{c}^{T} \bar{\hat{e}} + K_{o}^{T} (e - \hat{e}) \\ \hat{e} = C^{T} \bar{\hat{e}} \end{matrix}

(16)

where $K_{o} = [k_{1}^{o} k_{2}^{o} \dots k_{n}^{o}]^{T}$ is the observer gain vector, chosen such that the characteristic polynomial of $A - K_{o} C^{T}$ is strictly Hurwitz because (C, A) is observable. In the following, we will describe the adaptive law of the proposed controller and the convergence analysis of this controller.

Adaptive law

As discussed previously, the learning of the ESTN is simple, since only the output weights needs to be learned while all other weights are fixed during learning. Thus, the computational complexity of the ESTN training would be lower than most other recurrent and feedforward networks. We present the adaptation law of the output weight $W^{out}$ of the ESTN controller in the proposed scheme, since it is updated through minimization of a suitable cost function. We define the control error $\tilde{u}$ between the control law (3) and the estimated control law (12) associated with the output weights $W^{out}$ as

\tilde{u} (t) = u^{*} (t) - \hat{u} (t)

(17)

Herein, the cost function chosen is defined as

E (t) = \frac{1}{2} \tilde{u} (t)^{2}

(18)

Hence, the minimization of $E (t)$ through the gradient descent is given by

W^{out} (t + 1) = W^{out} (t) - μ \nabla_{W^{out} (t)} E (t)

(19)

where $μ$ is the step size, a small positive constant. The gradient $\nabla_{W^{out} (t)} E (t)$ can be calculated using both of the control error $\tilde{u}$ in (17) and the estimated control law in (12) such that

\nabla_{W^{out} (t)} E (t) = - \tilde{u} (t) υ (t)

(20)

The updating law of the output weights of the ESTN controller is then given by

W^{out} (t + 1) = W^{out} (t) + μ \tilde{u} (t) υ (t) .

(21)

Due to the fact that the ideal controller is not available, the ideal control error defined in (17) can not be computed. An estimated value denoted by ${\tilde{u}}_{est} (t)$ can be used instead in the adaptive law (21). In Mahmoud and Elshenawy (2015), a simple method is proposed to estimate the ideal control error with low computational cost compared to Belarbi and Chemachema (2007) and Chemachema (2012). This method supposes that the value ${\tilde{u}}_{est} (t)$ can be estimated directly as

{\tilde{u}}_{est} (t) = g_{1} e + g_{2} \overset{\cdot}{e}

(22)

where $e = y_{r} - x_{1}$ , $g_{1}$ and $g_{2}$ are small real values. The sign of both $g_{1}$ and $g_{2}$ will be selected as positive in case of positive control gain systems while it will be negative for negative control gain systems. Because both of $\tilde{u} (t)$ and ${\tilde{u}}_{est} (t)$ are scalers and (22) guarantees ${\tilde{u}}_{est} (t)$ has the same sign of $\tilde{u} (t)$ , it means

\tilde{u} (t) = S {\tilde{u}}_{est} (t)

(23)

where S is a positive scaling factor. Selecting this scaling factor will only affect the step size of the updating and can be involved in the step size constant $μ$ . Consequently, the adaptive laws of the controller can be rewritten as

W^{out} (t + 1) = W^{out} (t) + μ {\tilde{u}}_{est} (t) υ (t)

(24)

Convergence and stability analysis

In this subsection we will investigate the convergence of the ESTN controller and the stability conditions of the closed loop system. In order to analyse the convergence of the ESTN controller, we can use the convergence criterion described in the literature (Ku and Lee, 1995; Mahmoud, 2011; Selami and Musa, 2010).

By defining the discrete Lyapunov function

V_{c} (t) = E = \frac{1}{2} \tilde{u} (t)^{2}

(25)

The change of the Lyapunov function can be obtained as

\begin{matrix} Δ V_{c} (t) = \frac{1}{2} (\tilde{u} {(t + 1)}^{2} - \tilde{u} {(t)}^{2}) \\ = \frac{1}{2} (\tilde{u} (t + 1) - \tilde{u} (t)) (\tilde{u} (t + 1) + \tilde{u} (t)) \\ = \frac{1}{2} (Δ \tilde{u} (t)) (2 \tilde{u} (t) + Δ \tilde{u} (t)) \\ = Δ \tilde{u} (t) (\tilde{u} (t) + \frac{1}{2} Δ \tilde{u} (t)) \end{matrix}

(26)

where $Δ \tilde{u} (t)$ is the change of the control error and we can write it by the first-order Taylor series expansion in terms of the ESTN controller weights $W^{out} (t + 1)$ as

Δ \tilde{u} (t) ≅ \frac{\partial \tilde{u} (t)}{\partial W^{out} (t)} Δ W^{out} (t)

(27)

Using (12) and (17), the partial derivative $\partial \tilde{u} (t) / \partial W^{out} (t)$ is

\begin{matrix} \frac{\partial \tilde{u} (t)}{\partial W^{out} (t)} = - υ (t) \end{matrix}

(28)

The term $Δ W^{out} (t)$ can be expressed using (21) as

Δ W^{out} (t) = W^{out} (t + 1) - W^{out} (t) = μ \tilde{u} (t) υ (t)

(29)

By substituting (28) and (29) into (27) we obtain

Δ \tilde{u} (t) ≅ - μ \tilde{u} (t) υ (t)^{T} υ (t)

(30)

The change of the Lyapunov function $Δ V_{c} (t)$ can be rewritten by substituting (30) into (26) as

\begin{matrix} Δ V_{c} (t) = - μ \tilde{u} (t) υ {(t)}^{T} υ (t) (\tilde{u} (t) - \frac{1}{2} μ \tilde{u} (t) υ {(t)}^{T} υ (t)) \\ = - μ {\tilde{u}}^{2} (t) υ {(t)}^{T} υ (t) (1 - \frac{1}{2} μ υ {(t)}^{T} υ (t)) \end{matrix}

(31)

Then, the condition of $Δ V_{c} (t)$ is negative when

\begin{matrix} μ > 0, \frac{1}{2} μ υ {(t)}^{T} υ (t) < 1 \end{matrix}

(32)

Therefore, the convergence of the ESTN controller in the proposed output feedback control scheme is guaranteed if the range of the step-size $μ$ is

\begin{matrix} 0 < μ < \frac{2}{υ {(t)}^{T} υ (t)} \end{matrix}

(33)

To study the tracking error convergence for the closed loop system based on the proposed output feedback control method, the following Lemma is given.

Lemma 1.Consider the nonlinear SISO system expressed by (1). The ESTN controller (12) is designed with input vector $z (t) = [- {\bar{\hat{x}}}^{T}, y_{r}^{(n)}, K_{c}^{T} \bar{\hat{e}}]^{T}$ . If the initial values of all ESTN controller weights ( $W^{out}$ , W, $W^{in}$ ) are bounded and the tanh activation function is used in (11). The observer expressed in (16) is designed. Here, in the adaptive control scheme, the adaptive law (24) is given. As a result all the signals in the closed loop system are bounded and the tracking error converges exponentially to a bounded compact as $t \to \infty$ .

Proof. Define the following errors

\begin{matrix} \tilde{e} = \bar{e} - \bar{\hat{e}} = \bar{\hat{x}} - \bar{x} \\ \tilde{e} = e - \hat{e} = {\hat{x}}_{1} - x_{1} \end{matrix}

Subtracting (16) from (15), we can obtain the error dynamics for the output feedback control scheme such that

\begin{matrix} \overset{\cdot}{\tilde{e}} = A_{o} \tilde{e} + B (g (\bar{\hat{x}}) (u^{*} - \hat{u}) - d) \\ \tilde{e} = C^{T} \tilde{e} \end{matrix}

(34)

Using the control error (17), the former equation can be rewritten as

\begin{matrix} \overset{\cdot}{\tilde{e}} = A_{o} \tilde{e} + B (g (\bar{\hat{x}}) (\tilde{u}) - d) \\ \tilde{e} = C^{T} \tilde{e} \end{matrix}

(35)

where $A_{o} = A - K_{o} C^{T}$ . Consider the following Lyapunov function for the error dynamics in (35)

\begin{matrix} V_{o} = \frac{1}{2} {\tilde{e}}^{T} P_{o} \tilde{e} \end{matrix}

(36)

where $P_{o}$ is the solution of the Lyapunov equation

\begin{matrix} A_{o}^{T} P_{o} + P_{o} A_{o} = - Q_{o} \end{matrix}

(37)

where $Q_{o}$ is an arbitrary $n \times n$ positive definite matrix. The time derivative of $V_{o}$ with respect to time along the tracking error (35) is given by

\begin{matrix} {\overset{\cdot}{V}}_{o} = \frac{1}{2} {\overset{\cdot}{\tilde{e}}}^{T} P_{o} \tilde{e} + \frac{1}{2} {\tilde{e}}^{T} P_{o} \overset{\cdot}{\tilde{e}} = - \frac{1}{2} {\tilde{e}}^{T} Q_{o} \tilde{e} + {\tilde{e}}^{T} P_{o} B (g (\bar{\hat{x}}) \tilde{u} - d) \\ \leq - \frac{1}{2} {\tilde{e}}^{T} Q_{o} \tilde{e} + | {\tilde{e}}^{T} | | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |) \end{matrix}

(38)

Then (38) can be rewritten as

\begin{matrix} {\overset{\cdot}{V}}_{o} \leq - \frac{1}{2} λ_{min} (Q_{o}) | {\tilde{e}}^{2} | + | {\tilde{e}}^{T} | | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |) \\ \leq - \frac{1}{2} \frac{λ_{\min} (Q_{o})}{λ_{max} (P_{o})} λ_{max} (P_{o}) | {\tilde{e}}^{2} | + | {\tilde{e}}^{T} | | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |) \\ \leq - \frac{1}{2} \frac{λ_{min} (Q_{o})}{λ_{max} (P_{0})} {\tilde{e}}^{T} P_{o} \tilde{e} + | {\tilde{e}}^{T} | | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |) \\ \leq - \frac{1}{2} \frac{λ_{\min} (Q_{o})}{λ_{\max} (P_{o})} V_{o} + | {\tilde{e}}^{T} | | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |) \end{matrix}

(39)

where $λ_{min} (.)$ and $λ_{max} (.)$ are the smallest and largest eigenvalues of the defined matrices, respectively. Let $β_{o 1} = λ_{\min} (Q_{o}) / 2 λ_{\max} (P_{o})$ and $β_{o 2} = | P_{o} B | (| g (\bar{x}) \tilde{u} | + | d |)$ . Since $g (\bar{x})$ and $\tilde{u}$ are bounded, (39) can be written as

\begin{matrix} {\overset{\cdot}{V}}_{o} \leq - β_{o 1} V_{o} + β_{o 2} | {\tilde{e}}^{T} | \end{matrix}

(40)

which gives

\begin{matrix} V_{o} \leq \frac{β_{o 2} | {\tilde{e}}^{T} |}{β_{o 1}} + (V_{o} (0) - \frac{β_{o 2} | {\tilde{e}}^{T} |}{β_{o 1}}) e^{- β_{o 1} t} \end{matrix}

(41)

As $t \to \infty$ , $| V_{o} | \leq β_{o 2} | {\tilde{e}}^{T} | / β_{o 1}$ and $V_{o} \geq λ_{min} (P_{o}) {| ({\tilde{e}}^{T}) |}^{2}$ . Then we find

\begin{matrix} λ_{min} (P_{o}) {| {\tilde{e}}^{T} |}^{2} \leq V_{o} \leq \frac{β_{o 2} | {\tilde{e}}^{T} |}{β_{o 1}} \end{matrix}

As a result we have

\begin{matrix} | {\tilde{e}}^{T} | \leq \frac{1}{λ_{min} (P_{o})} \frac{β_{o 2}}{β_{o 1}} \end{matrix}

(42)

Therefore, the error dynamics of the output feedback control scheme converges exponentially to the bounded region

\begin{matrix} Ω_{o} (\tilde{e}) = {\tilde{e} | | {\tilde{e}}^{T} | \leq \frac{1}{λ_{min} (P_{o})} \frac{β_{o 2}}{β_{o 1}}} \end{matrix}

(43)

The overall scheme of the proposed observer-based echo-state network is depicted in Figure 2 and it can be described as follows.

Select the number of the reservoir of the ESTN (i.e. N). For choosing this value, a trade off between the computational complexity and the performance has to be made.

Initialize the weights W, $W^{in}$ and $W^{out}$ of the ESTN. They can be randomly chosen according to a normal distribution $~ N (0, 1)$ .

For the sake of the network convergence, normalize the matrix W by dividing it by its largest eigenvalue (i.e. $λ_{\max})$ .

Select the observer and feedback gain vectors $K_{o}$ , $K_{c}$ such that the matrices $A_{o} = A - K_{o} C^{T}$ and $A_{c} = A - B K_{c}^{T}$ are Hurwitz matrices, respectively.

Select the two parameters of the control error in (22) (i.e. $g_{1}, g_{2}$ ) as a small positive values to prevent the output response from having oscillation or overshoots.

Get the control law (11) and (12).

Obtain the control errors (22) and (23), and the control law (24).

Simulation results

This section presents the simulation results of the proposed algorithm for a class of unknown nonlinear dynamical systems. We consider two illustrative systems, the regulation problem of nonlinear mass-spring damper system and the control of a robot manipulator system. For each system, we performed two sets of simulations; one is to verify the performance of the proposed controller without any external disturbance and the second one is to test the controller when an external disturbance is imposed.

System 1

A nonlinear mass-spring-damper system is given to illustrate the effectiveness of the proposed procedure. The nonlinear model of this system is given by (Lam et al., 2001)

\begin{array}{l} {\overset{\cdot}{x}}_{1} = x_{2} \\ {\overset{\cdot}{x}}_{2} = f (x) + g (x) u (t) + d (t) \end{array}

(44)

where

\begin{matrix} f (x) = - x_{2} - 0.01 x_{1} - 0.01 x_{1}^{3} \\ g (x) = 1.4387 - 0.13 x_{2}^{2} \end{matrix}

where $x_{1}$ is the mass displacement, $u (t)$ is the force, and the external disturbance $d (t) = \sin (t)$ . The operating range of the states is assumed to be within the interval $[- 1.5, 1.5]$ . The initial states are $[1.5, 0]$ . For this system, the control objective is to regulate the mass at the origin of displacement $y_{r} (t) = 0$ . The initial values of the output weights of the network controller $W^{out} (0)$ are chosen randomly in the interval $[- 5, 5]$ and the number of the reservoir of the ESTN is selected as $N = 50$ . Both the state feedback gain and the observer gain vectors are chosen as $K_{c}^{T} =$ [10 10] and $K_{o}^{T} =$ [30 200], respectively.

Case study 1: At $d = 0$ , the tracking response of the mass displacement and the force applied to the system using the proposed controller are shown in Figures 3 and 4, respectively. The results of the simulation show that the proposed controller can regulate the system output within an acceptable time and the control signal is bounded and smooth.

Case study 2: We consider the external disturbance as $d (t) = \sin (t)$ , the simulation results are depicted in Figures 5 and 6. The results indicate a satisfied tracking performance with bounded closed loop signals in the presence of the disturbance.

System 2

Here, we apply the proposed direct adaptive controller for the tracking problem of one-link rigid robotic manipulator. This system is used in Lin and Peng (2004) and Chemachema (2012) to demonstrate the performance of the adaptive controllers suggested therein. The dynamic equation of this system is given by

\begin{matrix} m l^{2} \overset{\cdot\cdot}{q} + b \overset{\cdot}{q} + m l g \cos (q) = u \end{matrix}

(45)

where q is the angular position with initial values $q = 0.1$ , $\overset{\cdot}{q} = 0$ , u is the input torque, $m, l$ are the mass and the length of the link, respectively, and d is the external disturbance. The above equation can be written as

\begin{matrix} {\overset{\cdot}{x}}_{1} = x_{2} \\ {\overset{\cdot}{x}}_{2} = f (x) + g (x) u (t) + d (t) \end{matrix}

(46)

where $x_{1} = q$ , $x_{2} = \overset{\cdot}{q}$ , $f (x) = (- b / m l^{2}) x_{2} - (g_{v} / l) \cos (x_{1})$ , $g (x) = 1 / m l^{2}$ , and $d (t)$ is the external disturbance. The system parameters are chosen as $m = l = b = g = 1$ . The desired output of the system is given by the following reference model

\begin{array}{l} {\overset{\cdot}{x}}_{d 1} = x_{d 2} \\ {\overset{\cdot}{x}}_{d 2} = - 16 x_{d 1} - 8 x_{d 2} + r (t) \end{array}

(47)

where $[x_{d 1} (0), x_{d 2} (0)]^{T} = [0, 0]^{T}$ and $r (t)$ is a periodic rectangular signal. In this simulation, the state feedback gain and the observer gain vectors are chosen as $K_{c}^{T} =$ [40 4], $K_{o}^{T} =$ [9.3 86.38], respectively, and the ESTN controller has been designed similar to system (1). The external disturbance is assumed to be a square wave with amplitude $\pm 0.1$ with the same period of the periodic rectangular signal $r (t)$ .

Case study 1: At $d = 0$ , the simulation results are shown in Figures 7 to 9. The tracking performance of the states $x_{1}$ and $x_{2}$ are shown in Figures 7 and 8, respectively. Figure 9 illustrates the control signal response. We can see that the system states go quickly to the desired states and the control signal is bounded and smooth.

Case study 2: When the external disturbance described above is imposed to the system, the simulation results are illustrated in Figures 10 to 12. The figures indicate the satisfied tracking performance with bounded closed-loop system signals. We can conclude that the proposed observer-based echo state network controller can perform successful control despite the presence of the external disturbance.

Conclusion

In this paper, an observer-based echo-state neural control scheme is proposed for uncertain affine SISO nonlinear systems in the presence of bounded disturbances. To design the adaptive control law, no exact knowledge of the structure of the system nonlinearities is needed. In addition, the offline tuning of the ESTN controller weights is not required. The overall adaptive scheme guarantees that all signals involved are bounded and the output of the closed-loop system asymptotically tracks the desired output trajectory. Finally, the proposed scheme has been applied to control two examples of affine SISO nonlinear systems, the mass-spring nonlinear system and one-link rigid robotic manipulator system. The computer simulation results show that the observer-based direct adaptive ESTN controller can perform successful control and achieve the desired performance. In the future, an investigation of the proposed scheme for MIMO systems will be an interesting research topic in this field.

Footnotes

Declaration of Conflicting Interests

The authors declare that there is no conflict of interests.

Funding

This research received no specific grant from any funding agency in the public, commercial, or not-for-profit sectors.

References

Ahmed

Faouz

(2015) Centralized predictive controller design based on an uncertain neural network for multivariable non-linear systems. Transactions of the Institute of Measurement and Control 37(8): 952–960.

Artemis

Kostarigka

Rovithakis

(2012) Adaptive dynamic output feedback neural network control of uncertain MIMO nonlinear systems with prescribed performance. IEEE Transactions on Neural Networks and Learning Systems 23(1): 138–149.

Belarbi

Chemachema

(2007) Stable direct adaptive neural network controller with a fuzzy estimator of the control error for a class of perturbed nonlinear systems. IET Control Theory Applications 1(5): 1398–1404.

Castaneda

Esquivel

(2012) Decentralized neural identifier and control for nonlinear systems based on extended Kalman filter. Neural Networks 31: 81–87.

Chemachema

(2012) Output feedback direct adaptive neural network control for uncertain SISO nonlinear systems using a fuzzy estimator of the control error. Neural Networks 36: 25–34.

Chen

Khalil

(1995) Adaptive control of a class of nonlinear discrete-time systems neural networks. IEEE Transaction on Automatic Control 40(5): 791–801.

Chen

Lin

(1994) Adaptively controlling nonlinear continuous time systems using multilayer neural networks. IEEE Transaction on Automatic Control 39(6): 1306–1310.

Chen

How

(2010) Robust adaptive neural network control for a class of uncertain MIMO nonlinear systems with input nonlinearities. IEEE Transactions on Neural Networks 21(4): 796–812.

Chi–Hsu

Han–Leih

Tsung–Chilh

(2002) Direct adaptive fuzzy neural control with state observer supervisory controller for unknown nonlinear dynamical systems. IEEE Transactions on Fuzzy Systems 10(2): 39–49.

10.

Chien

Wang

Leu

. (2011) Robust adaptive controller design for a class of uncertain nonlinear systems using online TS fuzzy-neural modeling approach. IEEE Transaction on Systems Man, and Cybernetics-part B 41(2): 542–552.

11.

Chun–Fei

(2014) Adaptive backstepping elman-based neural control for unknown nonlinear systems. Neurocomputing 136: 170–179.

12.

Chun–Fei

Chin–Min

Ang–Bung

. (2012) Adaptive control for MIMO uncertain nonlinear systems using recurrent wavelet neural network. International Journal of Neural Systems 22(1): 37–50.

13.

El–Sousy

FFM

(2013) Intelligent optimal recurrent wavelet elman neural network control system for permanent-magnet synchronous motor servo drive. IEEE Transactions on Industrial Informatics 9(4): 1986–2003.

14.

Farhang–Boroujeny

(1998) Adaptive Filters: Theory and Applications. New York: John Wiley & Sons, Inc.

15.

Xie

Luo

(2013) Robust on-line nonlinear systems identification using multilayer dynamic neural networks with two-time scales. Neurocomputing 113: 16–26.

16.

Hang

Zhang

(1999) Adaptive neural network control of nonlinear systems by state and output feedback. IEEE Transaction on Systems Man, and Cybernetics-part B 29(6): 818–828.

17.

Guofa

Dongwu

Xuemei

(2015) Modified neural dynamic surface approach to output feedback of MIMO nonlinear systems. IEEE Transactions on Neural Networks and Learning Systems 26(2): 224–236.

18.

Jaeger

(2001) The echo state approach to analyzing and training recurrent neural networks. GMD Report 148. German National Research Institute for Computer Science, St. Augustin, Germany.

19.

Jaeger

(2007) Optimization and application of echo state networks with leaky integrator neurons. Neural Networks 20: 335–352.

20.

Jaeger

Haas

(2004) Harnessing nonlinearity: Predicting chaotic systems and saving energy in wireless communication. Science 304: 78–80.

21.

Jianhua

Qinmin

Jiangang

. (2015) An adaptive artificial neural network-based supply air temperature controller for air handling unit. Transactions of the Institute of Measurement and Control 37(9): 1118–1126.

22.

Lee

(1995) Diagonal recurrent development neural networks for dynamic systems control. IEEE Transactions on Neural Networks 6(1): 144–156.

23.

Lam

Leung

FHF

Tam

PKS

(2001) Non linear state feedback controller for nonlinear systems: stability analysis and design based on fuzzy plant model. IEEE Transactions on Fuzzy Systems 9(4): 657–661.

24.

Lee

Teng

(2000) Identification and control of dynamic systems using recurrent fuzzy neural networks. IEEE Transactions on Fuzzy Systems 8(4): 349–366.

25.

Jia

Liu

. (2014) Nonlinear adaptive control using multiple models and dynamic neural networks. Neurocomputing 136: 190–200.

26.

Yang

. (2011) Adaptive output feedback NN control of a class of discrete-time MIMO nonlinear systems with unknown control directions. IEEE Transactions on Systems, Man, and Cybernetics Part B 41(2): 507–517.

27.

Lin

Peng

(2004) Adaptive CMAC-based supervisory control for uncertain nonlinear systems. IEEE Transaction on Systems Man, and Cybernetics-part B 34(2): 1248–1260.

28.

Maass

Natschlager

Markram

(2002) Real time computing without stable states: A new framework for neural computation based on perturbations. Neural Computation 14(11): 2531–2560.

29.

Mahmoud

(2011) Adaptive control scheme based on the least squares support vector machine network. International Journal of Applied Mathematics and Computer Science 21(4): 685–696.

30.

Mahmoud

Elshenawy

(2015) Echo state neural network based state feedback control for SISO affine nonlinear systems. In: Proceedings of 1st IFAC conference on modeling, identification and control of nonlinear systems (MICNON-2015), Saint-Petersburg, Russia, 24–26 June 2015, pp. 364–369.

31.

Mandic

Chambers

(2001) Recurrent Neural Networks for Prediction: Learning Algorithms, Architectures and Stability. New York: John Wiley & Sons, Inc.

32.

Mark

Harris

(2007) Automatic speech recognition using a predictive echo state network classifier. Neural Networks 20: 414–423.

33.

Michael

Fub

. (2014) An adaptive recurrent neural-network controller using a stabilization matrix and predictive inputs to solve a tracking problem under disturbances. Neural Networks 49: 74–86.

34.

Minrui

Jian

Huosheng

. (2006) A novel linear recurrent neural network for multivariable system identification. Transactions of the Institute of Measurement and Control 28(2): 229–242.

35.

Narendra

Parthasarathy

(1990) Identification and control of dynamical systems using neural networks. IEEE Transactions on Neural Networks 1: 4–27.

36.

Ozdemir

Iskender

Ozg

(2011) Complex valued neural network with mobius activation function. Communications in Nonlinear Science and Numerical Simulation 16(12): 4698–4703.

37.

Pan

Wang

(2012) Model predictive control of unknown nonlinear dynamical systems based on recurrent neural networks. IEEE Transactions on Industrial Electronics 59(8): 3089–3101.

38.

Polycarpou

(1996) Stable adaptive neural control scheme for nonlinear systems. IEEE Transactions on Automatic Control 41(3): 447–451.

39.

Sanner

Slotine

(1992) Gaussian networks for direct adaptive control. IEEE Transactions on Neural Networks 3(5): 837–863.

40.

Selami

Musa

(2010) Stable modeling based control methods using a new RBF network. ISA Transactions 49: 510–518.

41.

Seong

Lee

(2013) Precise positioning of non smooth dynamic systems using fuzzy wavelet echo state networks and dynamic surface sliding mode control. IEEE Transactions on Industrial Electronics 60(11): 5124–5136.

42.

Seong

Lee

(2014) Fuzzy echo state neural networks and funnel dynamic surface control for prescribed performance of a nonlinear dynamic system. IEEE Transactions on Industrial Electronics 61(2): 1099–1112.

43.

Sridar

Khalil

(2000) Output feedback control of nonlinear systems using RBF neural networks. IEEE Transactions on Neural Networks 11(1): 69–79.

44.

Suping

Xizhen

(2015) Echo state network based dual adaptive control for trajectory tracking of wheeled mobile robot. In: The 5th annual IEEE international conference on cyber technology in automation. Shenyang, China, 8–12 June 2015, pp. 1223–1228.

45.

Tang

Sun

(2006) Neural network control of flexible-link manipulators using sliding mode. Neurocomputing 70: 288–295.

46.

Tonga

Han–Xiong

Wang

(2004) Observer-based adaptive fuzzy control for SISO nonlinear systems. Fuzzy Sets and Systems 148: 335–176.

47.

Townley

Ilchmann

Weiß

. (2000) Existence and learning of oscillations in recurrent neural networks. IEEE Transactions on Neural Networks 11(1): 205–214.

48.

Venayagamoorthy

(2007) Online design of an echo state network based wide area monitor of a multimachine power systems. Neural Networks 20: 404–413.

49.

Waegeman

Francis

Benjamin

(2012) Feedback control by online learning an inverse model. IEEE Transactions on Neural Networks and Learning Systems 23(10): 1637–1647.

50.

Wai

(2003) Tracking control based on neural network strategy for robot manipulator. Neurocomputing 51: 425–445.

51.

Wang

Dan

Zhouhua

(2014) Neural network based adaptive dynamic surface control for cooperative path following of marine surface vehicles via state and output feedback. Neurocomputing 133: 170–178.

52.

Wang

(1994) Adaptive Fuzzy System and Control: Design and Stability Analysis. NJ, USA: Prentice-Hall.

53.

Wang

Chai

Zhang

(2010) State observer-based adaptive fuzzy output-feedback control for a class of uncertain nonlinear systems. Information Sciences 180: 5029–5040.

54.

Wenkang

Chenxiao

Yun

(2014) Neural-network-based robot time-varying force control with uncertain manipulator-environment system. Transactions of the Institute of Measurement and Control 36(8): 999–1009.

55.

Williams

Zisper

(1990) An efficient gradient-based algorithm for on-line training of recurrent networks. Neural Computation 2(4): 490–501.

56.

Xie

Zhu

Zhao

. (2009) Nonlinear system identification using optimized dynamic neural network. Neurocomputing 72: 3277–3287.

57.

Lan

Principe

(2005) Direct adaptive control: An echo state network and genetic algorithm approach. In: Proceedings of IEEE international joint conference on neural networks, Montreal, Canada, 31 July–4 August 2005, pp. 1483–1486.

58.

Yih–Chuang

Tsu–Tian

Wei–Yen

(1999) Observer-based adaptive fuzzy neural control for unknown nonlinear dynamical systems. IEEE Transactions on Systems, Man, and Cybernetics-Part B 29(5): 538–591.

59.

Zhiwei

Min

(2007) Support vector echo-state machine for chaotic time-series prediction. IEEE Transactions on Neural Networks 18(2): 359–372.