Optimized dead-zone control based on a sliding-mode mechanism for a class of unknown nonlinear dynamic systems

Abstract

The article aims to propose an optimized tracking control for the high-order nonlinear dynamic system with the unknown dead zone of the controller. The dead zone refers to a control with zero behavior in some ranges, hence the system performance will be inevitably affected. Since the dead-zone phenomenon is frequently encountered in control systems, it is very necessary to consider the effect in optimized nonlinear control. Because the high-order system contains multiple system states, the optimized dead-zone control is designed by combining both reinforcement learning (RL) and sliding-mode control. Furthermore, an adaptive compensation of the remainder of the dead-zone function is added into the dead-zone input, then the optimized dead-zone control is yielded from the RL under identifier–actor–critic architecture, where the identifier can remove the requirement of known dynamic function. Finally, the effectiveness of this optimized control is proved by Lyapunov stability analysis and a simulation example.

Keywords

Optimal control unknown dead zone reinforcement learning sliding-mode mechanism nonlinear system

Introduction

Optimal control means to consume the least control source for fulfilling the control task (Hull, 2003). So far, it has already turned into a prevalent design principle in contemporary control systems. To find the optimal control of a nonlinear system, it involves the solution of the Hamilton–Jacobi–Bellman (HJB) equation (Abu-Khalaf and Lewis, 2004; Wen and Niu, 2022). But, because of the strong nonlinearity, getting the analytical solution of this equation is extremely challenging. To overcome the difficulty, reinforcement learning (RL) as an adaptive approximation means has been proven to be a workable method (Mozer et al., 2005). Up to now, it has been significantly researched and developed to address the optimal nonlinear control problems, such as Wen et al. (2018, 2019a, 2019b), Bian and Jiang (2022), and Yang et al. (2018).

Unfortunately, these optimal nonlinear controls using the traditional RL method, such as Wen et al. (2018, 2019a, 2019b), Bian and Jiang (2022), and Yang et al. (2018), still have many shortcomings. The most typical one is the high complexity in an algorithm, as a result, these optimal control methods are difficultly extended in theory and applied in engineering. Furthermore, the primary reason is that both actor and critic training laws for implementing RL are acquired through the negative gradient of the square of the approximated HJB equation. To alleviate the algorithm’s complexity, a simplified RL approach developed by Wen et al. (2020, 2021) was used to derive the adaptive updating laws from the negative gradient of a simple positive function, which is equivalent to the HJB equation, rather than the square of this equation. Subsequently, it is significantly extended and applied by combining with many control techniques proposed by Wen and Li (2022), Wen et al. (2022), Hua et al. (2017), Ji et al. (2022), and Shen et al. (2022).

Whereas, these mentioned optimal control methods still overlook the situation that the system performance is severely constrained by some stubborn nonlinear properties, such as dead zone, hysteresis, and saturation. Furthermore, the dead-zone problem of the controller, also known as the neutral zone or non-active zone, refers to the range of zero control signal, and it is frequently encountered in the control systems. For offsetting the effect of dead zone, scholars have proposed many control methods, such as those proposed by Zhou et al. (2006), Wang et al. (2004), Hua and Ding (2011), Hua et al. (2017), and Tong et al. (2015). In Zhou et al. (2006), a dead-zone inversion method has been developed for a nonlinear system. In Wang et al. (2004), a robust adaptive control method is explored to deal with this dead-zone problem. In Hua and Ding (2011), a control method is proposed to handle the multiple dead-zone inputs for the large-scale system. In Hua et al. (2017), the high-order stochastic multi-agent system with dead-zone input is discussed. In Tong et al. (2015), it is used to develop an observer control in the switching nonlinear system with dead zone. Furthermore, in Ren et al. (2021, 2022), an adaptive boundary control with input dead zones and external disturbances is studied. However, concerning the optimal nonlinear control, it still seldom involves the dead-zone problem. Particularly, for the optimized control of high-order nonlinear dynamic systems, the dead-zone problem will become more intractable because the control relates to the multiple state variables, so the complexity of RL is significantly raised.

To control a high-order system, sliding-mode control (SMC) is the most suitable strategy because SMC can smoothly steer multiple states by a sliding-mode variable (Chiang and Yang, 2006; Li et al., 2023; Ma and Li, 2020; Wen et al., 2023). Moreover, it has the benefits of quick global convergence, straightforward structure, minimal sensitivity to parameter fluctuations, and good tolerance to external disturbances. Inspired by the aforementioned, the article attempts to solve the optimization control problem of high-order nonlinear systems having unknown dead zones. The key contributions are listed below.

This optimized control method can effectively deal with dead-zone problems for high-order nonlinear systems. It is used to convert the dead-zone function into the linear form, then finding the dead-zone input yields the optimized control. Since the dead-zone input has an adaptive term to compensate for the remainder of the dead-zone function, the optimized method can be competent for various dead-zone systems.

Compared with the conventional method, this simplified optimal control algorithm can be performed easily. Since this optimal control obtains the RL from a straightforward positive function that is equivalent to the HJB equation rather than this equation’s square, it can make a substantial simplification of the algorithm.

This optimized control method does not ask for a complete dynamic acknowledgment. Since the RL algorithm integrates an adaptive identifier, it can remove the requirement of a known dynamic function.

Neural network (NN)

In the control community, NN can be a popular function approximator. Over a compact set $Ω \subset R^{n}$ , it is capable of approximating the continuous function $H (z) : R^{n} \to R$ via the following form:

H_{NN} (z) = W^{T} S (z),

(1)

where $W \in R^{p}$ is the NN weight with $p$ neurons; $S (z) = [S_{1} (z), \dots, S_{p} (z)]^{T} \in R^{p}$ is the basis function vector, which is defined by using the Gaussian function $S_{i} (z) = \exp [- {(z - τ_{i})}^{T} (z - τ_{i}) / 2 μ_{i}^{2}] \in R$ , of which $μ_{i} \in R$ is the width and $τ_{i} = [τ_{i 1}, \dots, τ_{in}]^{T} \in R^{n}$ is the centers of receptive fields.

Concerning this NN approximation (1), here must be an ideal weight vector $W^{*} \in R^{p}$ , which is formulated to be

W^{*} : = argmi n_{W \in R^{p}} {\sup_{W \in Ω_{z}} ∥ H (z) - W^{T} S (z) ∥} .

(2)

Consequently, the function $H (z)$ could re-described as

H (z) = W^{* T} S (z) + ϵ (z),

(3)

where $ϵ (z) \in R$ represents the approximation error, which satisfies $∥ ϵ (z) ∥ \leq υ$ , and $υ$ is a constant.

Main result

Problem statement

Regarding a high-order nonlinear system under the canonical dynamic form with relative agree $n$ ,

\begin{matrix} {\overset{\cdot}{x}}_{1} (t) = x_{2} (t), \\ {\overset{\cdot}{x}}_{2} (t) = x_{3} (t), \\ \dots \\ {\overset{\cdot}{x}}_{n} (t) = f (\bar{x}) + u, \end{matrix}

(4)

where $f (\bar{x}) \in R$ is a continuous nonlinear function, $\bar{x} (t) = [x_{1} (t), x_{2} (t), \dots, x_{n} (t)]^{T} \in R^{n}$ is the state vector, $u \in R$ is the control input variable, which is treated as the output of the following unknown dead zone.

The model of dead zone is formulated to be

u (t) = G (v) = {\begin{matrix} ϱ_{r} (v (t) - a_{r}), & v (t) > a_{r}, \\ 0, & a_{l} \leq v (t) \leq a_{r}, \\ ϱ_{l} (v (t) - a_{l}), & v (t) < a_{l}, \end{matrix}

(5)

where $v (t)$ is the input for this dead zone, $ϱ_{r} > 0$ and $ϱ_{l} > 0$ represent the dead-zone slopes, $a_{r} > 0$ and $a_{l} < 0$ stand for the dead-zone respective right and left break-points (Figure 1).

Figure 1.

Dead-zone model.

Assumption 1 (Wang et al., 2004; Yoo et al., 2009). The two dead-zone slopes of positive and negative regions are the same, that is, $ϱ_{r} = ϱ_{l} = ϱ$ .

By using Assumption 1, model (5) is re-described as

u (t) = G (v) = ϱ v (t) + g (v),

(6)

where

g (v) = {\begin{matrix} - ϱ a_{r}, & v (t) > a_{r}, \\ - ϱ v (t), & a_{l} \leq v (t) \leq a_{r}, \\ - ϱ a_{l}, & v (t) < a_{l} . \end{matrix}

(7)

From the above (7), it should be noticed that the function $g (v)$ is bounded, and it must exhibit a constant $\bar{g}$ that satisfies $| g (v) | \leq \bar{g}$ , where $\bar{g} = \max {ϱ a_{r}, - ϱ a_{l}}$ .

Remark 1. It is worth noting that the control of system (4) cannot be directly designed due to the existence of an unknown dead zone. However, from the output–input relation (5) of system control and dead zone, the control problem of system (4) is converted to find this dead-zone input. Furthermore, this dead zone (5) has a practical engineering background, such as hydraulic servo valve, servo motor (Wang et al., 2004), hence the dead-zone control is a significant research topic.

The control objective

In order to find the dead-zone input to yield the optimization tracking control for the high-order nonlinear dynamic system (4), so that (i) all control signals are semi-globally uniformly ultimately bounded (SGUUB) and (ii) the output state $x_{1} (t)$ can track the reference signal $ω_{r} (t)$ .

Lemma 1. (Wen et al., 2015). $N (t) \in R$ represents a positive continuous function, of which the initial value $N (0)$ is bounded. If it can hold $\overset{\cdot}{N} (t) \leq - ρ N (t) + ϑ$ , where both $ρ > 0$ and $ϑ > 0$ are constants, then there is an inequality that satisfies,

N (t) \leq e^{- ρ t} N (0) + \frac{ϑ}{ρ} (1 - e^{- ρ t}) .

(8)

Optimal control description

For the state tracking task, the desired reference trajectory can be denoted as $ω_{r} (t)$ , where its derivatives are assumed to be known and bounded. Then define $z_{1} (t) = x_{1} (t) - ω_{r} (t), z_{2} (t) = x_{2} (t) - {\overset{\cdot}{ω}}_{r} (t), \dots, z_{n} (t) = x_{n} (t) - ω_{r}^{(n - 1)} (t)$ as the tracking errors. From (4), the dynamic error is present as follows:

\begin{matrix} {\overset{\cdot}{z}}_{1} (t) = z_{2} (t), \\ {\overset{\cdot}{z}}_{2} (t) = z_{3} (t), \\ \dots \\ {\overset{\cdot}{z}}_{n} (t) = f (\bar{x}) + u - ω_{r}^{(n)} (t) . \end{matrix}

(9)

Define a sliding-mode variable as

\begin{matrix} s (t) = b_{1} z_{1} (t) + b_{2} {\overset{\cdot}{z}}_{1} (t) + \dots + b_{n - 1} {\overset{\cdot}{z}}_{1}^{(n - 2)} (t) + {\overset{\cdot}{z}}_{1}^{(n - 1)} (t) \\ = b_{1} z_{1} (t) + b_{2} z_{2} (t) + \dots + b_{n - 1} z_{n - 1} (t) + z_{n} (t) \\ = \sum_{i = 1}^{n - 1} b_{i} z_{i} (t) + z_{n} (t) \in R, \end{matrix}

(10)

where these coefficients $b_{1}, b_{2}, \dots, b_{n - 1}$ are chosen to make the eigenvalue polynomial $H (η) = η^{n - 1} + b_{n - 1} η^{n - 2} + \dots + b_{1}$ Hurwitz, that is, all its roots are located on the opened left-half plane.

Utilizing (9) and (10), the derivative of (10) is calculated to be

\begin{matrix} \overset{\cdot}{s} (t) = b_{1} {\overset{\cdot}{z}}_{1} (t) + b_{2} {\overset{\cdot}{z}}_{2} (t) + \dots + b_{n - 1} {\overset{\cdot}{z}}_{n - 1} (t) + {\overset{\cdot}{z}}_{n} (t) \\ = \sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) + u - ω_{r}^{(n)} (t) . \end{matrix}

(11)

Substituting (6) in (11) yields

\overset{\cdot}{s} (t) = \sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) + ϱ v (t) + g (v) - ω_{r}^{(n)} (t) .

(12)

The infinite horizon integral performance index for the dynamic (11) is considered as

V (0) = \int_{0}^{\infty} ϖ (s (ζ), u (s)) d ζ,

(13)

where $ϖ (s, u) = s^{2} (t) + u^{2}$ is the cost function.

Definition 1. (Admissible control; Vamvoudakis and Lewis, 2010). The control $u$ of (9) is considered admissible on the set $Ω$ , which is signified as $u \in Λ (Ω)$ , if $u$ holds $u (0) = 0$ , and is continuous, and stabilizes (9) on $Ω$ , even makes (13) finite.

The optimal control (Wen et al., 2021)

It is used to find an admissible control for (9), in order that it can minimize (13) for finishing the control assignment.

In view of (13), the performance index function can be produced as

V (s) = \int_{t}^{\infty} ϖ (s (ζ), u (s)) d ζ .

(14)

Let $u^{*}$ stand for the optimal dead-zone input, then, by replacing $u$ with $u^{*}$ in (14), it can yield the optimal performance index function:

\begin{matrix} V^{*} (s) = \int_{t}^{\infty} ϖ (s (ζ), u^{*} (s)) d ζ \\ = mi n_{u \in Ψ (Ω)} (\int_{t}^{\infty} ϖ (s (ζ), u (s)) d ζ), \end{matrix}

(15)

where $Ω$ is an already established compact set.

Along (11), computing the derivative of both sides of (15), thus it can obtain the HJB equation:

\begin{matrix} H (s, u^{*}, d V^{*} (s) / ds) = s^{2} (t) + u^{* 2} \\ + \frac{d V^{*}}{ds} (\sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) + u^{*} - ω_{r}^{(n)} (t)) = 0 . \end{matrix}

(16)

As stated previously, $u^{*}$ is uniquely correlated to (15), thus it can be requested as the only solution to hold above (16). Hence, $u^{*}$ can be obtained from $\partial H (s, u^{*}, d V^{*} (s) / ds) /$ $\partial u^{*} = 0$ as

u^{*} = - \frac{1}{2} \frac{d V^{*} (s)}{ds} .

(17)

From (6), the dead-zone input $v^{*}$ corresponding to $u^{*}$ can be represented to be

v^{*} = - \frac{1}{2 ϱ} \frac{d V^{*} (s)}{ds} - \frac{g (v^{*})}{ϱ} .

(18)

Nevertheless, the unknown terms $g (v^{*}) / ϱ$ and $d V^{*} (s) / ds$ are the obstacles to put the dead-zone input (18) into action. To achieve the optimal control, on the one hand, it has to consider to compensate the unknown term $g (v^{*}) / ϱ$ in this optimal control; on the other hand, this term $d V^{*} (s) / ds$ is hoped to gain by calculating the following HJB equation formed by putting (17) into (16):

\begin{matrix} H (s, u^{*}, d V^{*} (s) / ds) = s^{2} (t) - \frac{1}{4} {(\frac{d V^{*} (s)}{ds})}^{2} + \frac{d V^{*} (s)}{ds} \\ \times (\sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) - ω_{r}^{(n)} (t)) = 0 . \end{matrix}

(19)

However, due to the strong nonlinearity, the analytical solution of (19) is very challenging to obtain. So, the RL strategy is taken into consideration to determine the HJB equation’s approximated solution, and the adaptive approximation is taken into consideration to compensate $g (v^{*}) / ϱ$ .

RL design

To facilitate analysis and achieve tracking goal, we adopt a scheme of splitting $(1 / ϱ) (d V^{*} (s) / ds)$ into the following form:

\frac{1}{ϱ} \frac{d V^{*} (s)}{ds} = 2 {\bar{k}}_{u} s (t) + \frac{2 F (\bar{x}, \bar{z})}{ϱ} + V^{0} (s, \bar{z}),

(20)

where ${\bar{k}}_{u} > 0$ is the designed parameter, $F (\bar{x}, \bar{z}) = \sum_{i = 1}^{n - 1} b_{i} z_{i + 1}$ $(t) + f (\bar{x})$ and $V^{0} (s, \bar{z}) = - 2 {\bar{k}}_{u} s (t) - (2 F (\bar{x}, \bar{z}) / ϱ) + (1 / ϱ)$ $(d V^{*} (s) / ds) \in R$ , of which $\bar{z} = [z_{1}, \dots, z_{n}]^{T} \in R^{n}$ .

According to (6), the dead-zone input associated with $u^{*}$ is

v^{*} = - {\bar{k}}_{u} s (t) - \frac{F (\bar{x}, \bar{z})}{ϱ} - \frac{1}{2} V^{0} (s, \bar{z}) - \frac{g (v^{*})}{ϱ} .

(21)

Because the unknown terms $F (\bar{x}, \bar{z}) / ϱ$ and $V^{0} (s, \bar{z})$ are continuous, they are reformulated as the following forms by using the NN approximations:

\frac{F (\bar{x}, \bar{z})}{ϱ} = W_{F}^{* T} S_{F} (\bar{x}, \bar{z}) + ϵ_{F} (\bar{x}, \bar{z}),

(22)

V^{0} (s, \bar{z}) = W_{V}^{* T} S_{V} (s, \bar{z}) + ϵ_{V} (s, \bar{z}),

(23)

where $W_{F}^{*} \in R^{q}$ and $W_{V}^{*} \in R^{p}$ are the ideal NN weight matrices, $S_{F} (\bar{x}, \bar{z}) \in R^{q}$ and $S_{V} (s, \bar{z}) \in R^{p}$ are the basis function vectors, $ϵ_{F} (\bar{x}, \bar{z}) \in R$ and $ϵ_{V} (s, \bar{z}) \in R$ are two approximation errors that are bounded, and $q$ , $p$ are the number of neurons.

Inserting (22) and (23) into (20) and (21) yields

\frac{1}{ϱ} \frac{d V^{*} (s)}{ds} = 2 {\bar{k}}_{u} s (t) + 2 W_{F}^{* T} S_{F} (\bar{x}, \bar{z}) + W_{V}^{* T} S_{V} (s, \bar{z}) + ϵ,

(24)

v^{*} = - {\bar{k}}_{u} s (t) - W_{F}^{* T} S_{F} (\bar{x}, \bar{z}) - \frac{1}{2} W_{V}^{* T} S_{V} (s, \bar{z}) - \frac{g (v^{*})}{ϱ} - \frac{1}{2} ϵ,

(25)

where $ϵ = 2 ϵ_{F} + ϵ_{V}$ .

Nevertheless, the ideal weights $W_{F}^{*}$ , $W_{V}^{*}$ and the term $g (v^{*}) / ϱ$ are unknown, so the dead-zone input (25) is not available. To solve it, this article first establishes RL based on equations (22), (24) and (25) in the identifier–critic–actor architecture, and then creates the adaptive approximation method for the boundedness $β^{*} = {\bar{g}}^{2} / ϱ^{2}$ of term $g (v^{*}) / ϱ$ .

In accordance with (22), the identifier NN is established for estimating the unknown dynamic of the control as

\frac{\hat{F} (\bar{x}, \bar{z})}{ϱ} = {\hat{W}}_{F}^{T} S_{F} (\bar{x}, \bar{z}),

(26)

where $\hat{F} (\bar{x}, \bar{z}) / ϱ$ is the identifier output, ${\hat{W}}_{F} \in R^{q}$ is the identifier NN weight and has the following weight update law:

{\overset{\cdot}{\hat{W}}}_{F} (t) = S_{F} (\bar{x}, \bar{z}) s (t) - γ {\hat{W}}_{F} (t),

(27)

where $γ > 0$ is a designed constant.

In accordance with (24), the critic NN is established for assessing the performance of control as

\frac{1}{ϱ} \frac{d {\hat{V}}^{*} (s)}{ds} = 2 {\bar{k}}_{u} s (t) + 2 {\hat{W}}_{F}^{T} (t) S_{F} (\bar{x}, \bar{z}) + {\hat{W}}_{c}^{T} (t) S_{V} (s, \bar{z}),

(28)

where $(d {\hat{V}}^{*} (s) / ds) \in R$ is the estimate of $d V^{*} (s) / ds$ , ${\hat{W}}_{c} \in R^{p}$ is the critic NN weight and its weight update law listed below,

{\overset{\cdot}{\hat{W}}}_{c} (t) = - {\bar{k}}_{c} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) - \frac{1}{2} S_{V} (s, \bar{z}) s (t),

(29)

where ${\bar{k}}_{c} > 0$ , $σ > 0$ , and $I_{m}$ represent the critic design parameter, the design parameter and the identity matrix, respectively.

In accordance with (25), the actor NN is set up to execute the control action as

{\hat{v}}^{*} = - {\bar{k}}_{u} s (t) - {\hat{W}}_{F}^{T} S_{F} (\bar{x}, \bar{z}) - \frac{1}{2} {\hat{W}}_{a}^{T} S_{V} (s, \bar{z}) - \frac{1}{2} s (t) \hat{β} (t),

(30)

where ${\hat{W}}_{a} \in R^{p}$ is the actor NN weight, $\hat{β} (t) \in R$ is an adaptive estimation of the unknown constant $β^{*} = {\bar{g}}^{2} / ϱ^{2}$ , and ${\hat{W}}_{a}$ and $\hat{β} (t)$ are updated by the laws:

{\overset{\cdot}{\hat{W}}}_{a} (t) = - {\bar{k}}_{a} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) ({\hat{W}}_{a} (t) - {\hat{W}}_{c} (t)),

(31)

\dot{\hat{β}} (t) = \frac{1}{2} s^{2} (t) - κ \hat{β} (t),

(32)

where ${\bar{k}}_{a} > 0$ is the actor design parameter, $κ > 0$ is a designed constant.

Theory with proof

Theorem 1. To the high-order nonlinear system (4) with the bounded initial value. If the dead-zone input is used to obtain the optimized control by the suggested identifier–critic–actor RL of (26), (28), and (30) with the update laws (27), (29), and (31), the designed parameters are chosen to meet the requirements listed below:

{\bar{k}}_{u} > \frac{3}{2}, {\bar{k}}_{a} > \frac{1}{2}, {\bar{k}}_{c} > {\bar{k}}_{a} .

(33)

The following goals can be accomplished:

all errors $\bar{z} (t)$ , ${\tilde{W}}_{F} (t)$ , ${\tilde{W}}_{c} (t)$ , ${\tilde{W}}_{a} (t)$ , and $\tilde{β}$ are SGUUB, where $\bar{z} (t) = [z_{1}, \dots, z_{n}]^{T}$ , ${\tilde{W}}_{F} (t) = {\hat{W}}_{F} (t) - W_{F}^{*}$ , ${\tilde{W}}_{c} (t) = {\hat{W}}_{c} (t) - W_{V}^{*}$ , ${\tilde{W}}_{a} (t) = {\hat{W}}_{a} (t) - W_{V}^{*}$ , and $\tilde{β} = \hat{β} - β^{*}$ ;

all tracking errors $z_{1} (t), \dots, z_{n} (t)$ converge to a small zero neighborhood.

Proof. Discuss the candidate of the Lyapunov function to be

\begin{matrix} L (t) = \frac{1}{2 ϱ} s^{2} (t) + \frac{1}{2} {\tilde{W}}_{F}^{T} (t) {\tilde{W}}_{F} (t) + \frac{1}{2} {\tilde{W}}_{c}^{T} (t) {\tilde{W}}_{c} (t) \\ + \frac{1}{2} {\tilde{W}}_{a}^{T} (t) {\tilde{W}}_{a} (t) + \frac{1}{2} {\tilde{β}}^{2} (t) . \end{matrix}

(34)

□

Along (12), (27), (29), (31), and (32), the derivative of (34) is figured out as

\begin{matrix} \overset{\cdot}{L} (t) = \frac{1}{ϱ} s (t) (\sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) + ϱ v (t) + g (v) - ω_{r}^{(n)} (t)) \\ + {\tilde{W}}_{F}^{T} (t) (S_{F} (\bar{x}, \bar{z}) s (t) - γ {\hat{W}}_{F} (t)) \\ - {\tilde{W}}_{c}^{T} (t) ({\bar{k}}_{c} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} + \frac{1}{2} S_{V} (s, \bar{z}) s (t)) \\ - {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) \\ + σ I_{m}) ({\hat{W}}_{a} (t) - {\hat{W}}_{c} (t)) + \tilde{β} (t) (\frac{1}{2} s^{2} (t) - κ \hat{β} (t)) . \end{matrix}

(35)

Inserting (30) into (35) yields

\begin{matrix} \overset{\cdot}{L} (t) = \frac{1}{ϱ} s (t) (\sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}) + ϱ (- {\bar{k}}_{u} s (t) - \hat{W} S_{F}^{T} S_{F} (\bar{x}, \bar{z}) \\ - \frac{1}{2} {\hat{W}}_{a}^{T} S_{V} (s, \bar{z}) - \frac{1}{2} s (t) \hat{β} (t)) + g (v^{*}) - ω_{r}^{(n)} (t)) \\ + {\tilde{W}}_{F}^{T} (t) (S_{F} (\bar{x}, \bar{z}) s (t) - γ {\hat{W}}_{F} (t)) - {\tilde{W}}_{c}^{T} (t) ({\bar{k}}_{c} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) \\ + σ I_{m}) {\hat{W}}_{c} + \frac{1}{2} S_{V} (s, \bar{z}) s (t)) - {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) ({\hat{W}}_{a} (t) - {\hat{W}}_{c} (t)) \\ + \tilde{β} (t) (\frac{1}{2} s^{2} (t) - κ \hat{β} (t)) . \end{matrix}

(36)

Based on the NN approximation (22) of function $(1 / ϱ)$ $F (\bar{x}, \bar{z}) = (1 / ϱ) (\sum_{i = 1}^{n - 1} b_{i} z_{i + 1} (t) + f (\bar{x}))$ , the above (36) is rewritten as

\begin{matrix} \overset{\cdot}{L} (t) = - {\bar{k}}_{u} s^{2} (t) + s (t) W_{F}^{* T} S_{F} (\bar{x}, \bar{z}) - s (t) {\hat{W}}_{F}^{T} S_{F} (\bar{x}, \bar{z}) \\ + s (t) ϵ_{F} - \frac{1}{2} s (t) {\hat{W}}_{a}^{T} (t) S_{V} (s, \bar{z}) - \frac{1}{2} s (t) {\tilde{W}}_{c}^{T} (t) \\ \times S_{V} (s, \bar{z}) - \frac{1}{2} s^{2} (t) \hat{β} (t) + s (t) \frac{g (v^{*})}{ϱ} - \frac{1}{ϱ} s (t) ω_{r}^{(n)} \\ + {\tilde{W}}_{F}^{T} (t) S_{F} (\bar{x}, \bar{z}) s (t) - γ {\tilde{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) - {\bar{k}}_{c} {\tilde{W}}_{c}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) - {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) ({\hat{W}}_{a} (t) - {\hat{W}}_{c} (t)) \\ + \frac{1}{2} \tilde{β} (t) s^{2} (t) - κ \tilde{β} (t) \hat{β} (t) . \end{matrix}

(37)

From ${\tilde{W}}_{c} (t) = {\hat{W}}_{c} (t) - W_{V}^{*}$ and ${\tilde{W}}_{a} (t) = {\hat{W}}_{a} (t) - W_{V}^{*}$ , the following details exist:

\begin{matrix} - \frac{1}{2} s (t) {\hat{W}}_{a}^{T} (t) S_{V} (s, \bar{z}) - \frac{1}{2} s (t) {\tilde{W}}_{c}^{T} (t) S_{V} (s, \bar{z}) \\ = - \frac{1}{2} s (t) {\tilde{W}}_{a}^{T} (t) S_{V} (s, \bar{z}) - \frac{1}{2} s (t) {\hat{W}}_{c}^{T} (t) S_{V} (s, \bar{z}), \end{matrix}

(38)

{\hat{W}}_{a} (t) - {\hat{W}}_{c} (t) = {\hat{W}}_{a} (t) - W_{V}^{*} + W_{V}^{*} - {\hat{W}}_{c} (t) = {\tilde{W}}_{a} (t) - {\tilde{W}}_{c} (t) .

(39)

Using ${\tilde{W}}_{F} (t) = {\hat{W}}_{F} (t) - W_{F}^{*}$ and inserting (38) and (39) into (37) yields

\begin{matrix} \overset{\cdot}{L} (t) = - {\bar{k}}_{u} s^{2} (t) - \frac{1}{2} s (t) {\tilde{W}}_{a}^{T} (t) S_{V} (s, \bar{z}) - \frac{1}{2} s (t) {\hat{W}}_{c}^{T} (t) \\ \times S_{V} (s, \bar{z}) + s (t) ϵ_{F} - \frac{1}{2} s^{2} (t) \hat{β} (t) + s (t) \frac{g (v^{*})}{ϱ} \\ - \frac{1}{ϱ} s (t) ω_{r}^{(n)} - γ {\tilde{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) - {\bar{k}}_{c} {\tilde{W}}_{c}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) - {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) ({\tilde{W}}_{a} (t) - {\tilde{W}}_{c} (t)) \\ + \frac{1}{2} \tilde{β} (t) s^{2} (t) - κ \tilde{β} (t) \hat{β} (t) . \end{matrix}

(40)

Using Young’s inequalities, we can obtain the following facts:

s (t) ϵ_{F} \leq \frac{1}{2} s^{2} (t) + \frac{1}{2} ϵ_{F}^{2},

(41)

- \frac{1}{ϱ} s (t) ω_{r}^{(n)} (t) \leq \frac{s^{2} (t)}{2} + \frac{{(ω_{r}^{(n)})}^{2}}{2 ϱ^{2}},

(42)

\frac{g (v^{*})}{ϱ} s (t) \leq \frac{1}{2} \frac{{\bar{g}}^{2}}{ϱ^{2}} s^{2} (t) + \frac{1}{2} = \frac{1}{2} β^{*} s^{2} (t) + \frac{1}{2},

(43)

\begin{matrix} - \frac{1}{2} s (t) {\tilde{W}}_{a}^{T} (t) S_{V} (s, \bar{z}) \leq \frac{1}{4} s^{2} (t) + \frac{1}{4} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) \\ + σ I_{m}) {\tilde{W}}_{a} (t), \end{matrix}

(44)

\begin{matrix} - \frac{1}{2} s (t) {\hat{W}}_{c}^{T} (t) S_{V} (s, \bar{z}) \leq \frac{1}{4} s^{2} (t) + \frac{1}{4} {\hat{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) \\ + σ I_{m}) {\hat{W}}_{c} (t) . \end{matrix}

(45)

Utilizing the results from above, (40) can turn into

\begin{matrix} \overset{\cdot}{L} (t) \leq - ({\bar{k}}_{u} - \frac{3}{2}) s^{2} (t) - γ {\tilde{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) - {\bar{k}}_{c} {\tilde{W}}_{c}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) - ({\bar{k}}_{a} - \frac{1}{4}) \\ \times {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{a} (t) \\ + {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) \\ + \frac{1}{4} {\hat{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) \\ - \frac{1}{2} s^{2} (t) \hat{β} (t) + \frac{1}{2} β^{*} s^{2} (t) + \frac{1}{2} \tilde{β} (t) s^{2} (t) \\ - κ \tilde{β} (t) \hat{β} (t) + \frac{1}{2} ϵ_{F}^{2} + \frac{{(ω_{r}^{(n)})}^{2}}{2 ϱ^{2}} + \frac{1}{2} . \end{matrix}

(46)

Based on $\tilde{β} (t) = \hat{β} (t) - β^{*}$ , (46) is adjusted to be

\begin{matrix} \overset{\cdot}{L} (t) \leq - ({\bar{k}}_{u} - \frac{3}{2}) s^{2} (t) - γ {\tilde{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) - {\bar{k}}_{c} {\tilde{W}}_{c}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) - ({\bar{k}}_{a} - \frac{1}{4}) \\ \times {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{a} (t) \\ + {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) \\ + \frac{1}{4} {\hat{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) \\ - κ \tilde{β} (t) \hat{β} (t) + \frac{1}{2} ϵ_{F}^{2} + \frac{{(ω_{r}^{(n)})}^{2}}{2 ϱ^{2}} + \frac{1}{2} . \end{matrix}

(47)

From ${\tilde{W}}_{c} (t) = {\hat{W}}_{c} (t) - W_{V}^{*}$ , ${\tilde{W}}_{F} (t) = {\hat{W}}_{F} (t) - W_{F}^{*}$ and $\tilde{β} (t) = \hat{β} (t) - β^{*}$ , we can obtain

\begin{matrix} - {\bar{k}}_{c} {\tilde{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) \\ = - \frac{{\bar{k}}_{c}}{2} {\tilde{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) \\ - \frac{{\bar{k}}_{c}}{2} {\hat{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\hat{W}}_{c} (t) \\ + \frac{{\bar{k}}_{c}}{2} W_{V}^{* T} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) W_{V}^{*}, \end{matrix}

(48)

\begin{matrix} - γ {\tilde{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) = - \frac{γ}{2} {\tilde{W}}_{F}^{T} (t) {\tilde{W}}_{F} (t) \\ - \frac{γ}{2} {\hat{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) + \frac{γ}{2} W_{F}^{* T} W_{F}^{*}, \end{matrix}

(49)

- κ \tilde{β} (t) \hat{β} (t) = - \frac{κ}{2} {\tilde{β}}^{2} (t) - \frac{κ}{2} {\hat{β}}^{2} (t) + \frac{κ}{2} β^{* 2} .

(50)

Furthermore, based on Young’s inequality, we can derive

\begin{matrix} {\bar{k}}_{a} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) \\ \leq \frac{{\bar{k}}_{a}}{2} {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{a} (t) \\ + \frac{{\bar{k}}_{a}}{2} {\tilde{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) . \end{matrix}

(51)

Using above (48), (49), (50), and (51), (47) will become

\begin{matrix} \overset{\cdot}{L} (t) \leq - ({\bar{k}}_{u} - \frac{3}{2}) s^{2} (t) - \frac{γ}{2} {\tilde{W}}_{F}^{T} (t) {\tilde{W}}_{F} (t) \\ - \frac{γ}{2} {\hat{W}}_{F}^{T} (t) {\hat{W}}_{F} (t) - (\frac{{\bar{k}}_{c}}{2} - \frac{{\bar{k}}_{a}}{2}) {\tilde{W}}_{c}^{T} (t) \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{c} (t) - (\frac{{\bar{k}}_{a}}{2} - \frac{1}{4}) \\ \times {\tilde{W}}_{a}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) {\tilde{W}}_{a} (t) \\ - (\frac{{\bar{k}}_{c}}{2} - \frac{1}{4}) {\hat{W}}_{c}^{T} (t) (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) \\ \times {\hat{W}}_{c} (t) - \frac{κ}{2} {\tilde{β}}^{2} (t) - \frac{κ}{2} {\hat{β}}^{2} (t) + \frac{{\bar{k}}_{c}}{2} W_{V}^{* T} \\ \times (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) W_{V}^{*} \\ + \frac{γ}{2} W_{F}^{* T} W_{F}^{*} + \frac{κ}{2} {β^{*}}^{2} + \frac{1}{2} ϵ_{F}^{2} + \frac{{(ω_{r}^{(n)})}^{2}}{2 ϱ^{2}} + \frac{1}{2} . \end{matrix}

(52)

Depending on (33), the inequality (52) may shift as

\begin{matrix} \overset{\cdot}{L} (t) \leq - ({\bar{k}}_{u} - \frac{3}{2}) s^{2} (t) - \frac{γ}{2} {\tilde{W}}_{F}^{T} (t) {\tilde{W}}_{F} (t) \\ - (\frac{{\bar{k}}_{c}}{2} - \frac{{\bar{k}}_{a}}{2}) σ {\tilde{W}}_{c}^{T} (t) {\tilde{W}}_{c} (t) \\ - (\frac{{\bar{k}}_{a}}{2} - \frac{1}{4}) σ {\tilde{W}}_{a}^{T} (t) {\tilde{W}}_{a} (t) \\ - \frac{κ}{2} {\tilde{β}}^{2} (t) + C (t), \end{matrix}

(53)

where $C (t) = ({\bar{k}}_{c} / 2) W_{V}^{* T} (S_{V} (s, \bar{z}) S_{V}^{T} (s, \bar{z}) + σ I_{m}) W_{V}^{*} + (γ / 2)$ $W_{F}^{* T} W_{F}^{*} + \frac{κ}{2} {β^{*}}^{2} + (1 / 2) ϵ_{F}^{2} + ({(ω_{r}^{(n)})}^{2} / 2 ϱ^{2}) + (1 / 2)$ is con- strained by a constant $ϑ$ , that is, $| C (t) | \leq ϑ$ .

Subsequently, letting $ρ = \min {2 ({\bar{k}}_{u} - (3 / 2)) ϱ, ({\bar{k}}_{c} - {\bar{k}}_{a}) σ,$ $({\bar{k}}_{a} - (1 / 2)) σ, γ, κ}$ , (53) can be changed to

\overset{\cdot}{L} (t) \leq - ρ L (t) + ϑ .

(54)

By using Lemma 1, we can obtain

L (t) \leq e^{- ρ t} L (0) + \frac{ϑ}{ρ} (1 - e^{- ρ t}) .

(55)

Obtaining (55) implies that all error signals are SGUUB, in the meantime, by picking designed parameters large enough, the tracking error $\bar{z} (t)$ is directed toward a small zero neighborhood.

Simulation example

The example of a second-order nonlinear system simulation used in this section is below:

\begin{matrix} {\overset{\cdot}{x}}_{1} (t) = x_{2} (t), \\ {\overset{\cdot}{x}}_{2} (t) = u + {(1 + x_{1} (t))}^{2} + x_{2}^{2} (t), \end{matrix}

(56)

where $x_{1} (t)$ , $x_{2} (t)$ , and $u$ represent the position state, velocity state, and dead-zone output, respectively. And using $x_{1} (0) = 2$ and $x_{2} (0) = 3$ as the initial states. The dead-zone parameters in this simulation are $a_{r} = 0.6$ , $a_{l} = - 0.7$ , and $ϱ = 1$ .

When the time function $ω_{r} = 5 \sin (t)$ is used to define the reference trajectory, the tracking errors are produced as

\begin{matrix} z_{1} (t) = x_{1} (t) - 5 \sin (t), \\ z_{2} (t) = x_{2} (t) - 5 \cos (t) . \end{matrix}

(57)

In accordance with (10), the sliding-mode variable is designed as $s (t) = 3 z_{1} (t) + z_{2} (t)$ .

The three identifier, critic, and actor NNs are designed with $12$ neurons, and the centers $τ_{i}$ of Gaussian functions in the basis function vectors are uniformly distributed from $- 6$ to $6$ and the width $μ_{i} = 1$ . Furthermore, the RL parameters of this identifier, critic, actor and adaptive training laws correspond to (27), (29), (31), and (32) are set to $γ = 0.06$ , ${\bar{k}}_{c} = 0.7$ , ${\bar{k}}_{a} = 0.6$ , $σ = 0.04$ , and $κ = 1.3$ . Concerning the optimized dead-zone input corresponding to (30), the parameter is set to be ${\bar{k}}_{u} = 30$ . The initial values are set as $W_{c} (0) = [1.6, \dots, 1.6] \in R^{12 \times 1}$ , $W_{a} (0) = [1.8, \dots, 1.8] \in R^{12 \times 1}$ , $W_{F} (0) = [2.5, \dots, 2.5] \in R^{12 \times 1}$ , and $\hat{β} (0) = 1.2$ .

Figures 2 to 6 exhibit that the simulation results are ideal. Figure 2 displays the tracking performances. Figure 3 demonstrates the two tracking errors $z_{1} (t), z_{2} (t)$ , which are convergent to zero, and it indicates that these system states $x_{1} (t), x_{2} (t)$ can track the reference signal $ω_{r} (t), {\overset{\cdot}{ω}}_{r} (t)$ . Figures 4 to 6 demonstrate the identifier, critic, and actor NN weights and the adaptive parameter $\hat{β} (t)$ to be bounded. The cost function $ϖ (s, u)$ is shown in Figure 6. On the basis of Figures 2 to 6, all errors $\bar{z} (t)$ , ${\tilde{W}}_{F} (t)$ , ${\tilde{W}}_{c} (t)$ , ${\tilde{W}}_{a} (t)$ , and $\tilde{β}$ are SGUUB. It indicates that the control objectives can be well accomplished by the proposed optimized method.

Figure 2.

The tracking performance.

Figure 3.

The tracking errors.

Figure 4.

The norms of critic and actor NN weight.

Figure 5.

The identifier neural network (NN) weight and adaptive dead-zone parameter.

Figure 6.

The cost function.

Conclusion

This article develops an optimized tracking control to handle the dead-zone problem for the high-order nonlinear canonical system. To find the dead-zone input to output this optimized control, the RL and SMC strategies are combined. By adding an adaptive identifier into the RL design, the optimized control applied to an unknown dynamic system is put into action. Meanwhile, to solve the dead-zone problem more effectively, an adaptive approximation term is introduced into the RL to compensate for the unknown remainder of the dead-zone function. Finally, the optimal control approach has been shown to be capable of achieving control goals in line with the Lyapunov stability theory. Simulation results further validate the effectiveness of this approach.

Footnotes

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article:: This work is supported in part by the National Natural Science Foundation of China under grant nos. 62073045 and 61973185, and in part by the Natural Science Foundation of Shandong Province under grant nos. ZR2021MF088 and ZR2020MF097, and in part by the Development Plan of Young Innovation Team in Colleges and Universities of Shandong Province under grant no. 2019KJN011.

Data availability

All data included in this study are available upon request by contact with the corresponding author.

ORCID iDs

Shuaihua Ma

Guoxing Wen

References

Abu-Khalaf

Lewis

(2004) Nearly optimal state feedback control of constrained nonlinear systems using a neural networks HJB approach. Annual Reviews in Control 28: 239–251.

Bian

Jiang

(2022) Reinforcement learning and adaptive optimal control for continuous-time nonlinear systems: A value iteration approach. IEEE Transactions on Neural Networks and Learning Systems 33: 2781–2790.

Chiang

Yang

(2006) Robust adaptive fuzzy sliding mode control for a class of uncertain nonlinear systems with unknown dead-zone. In: 2006 IEEE international conference on fuzzy systems, Vancouver, BC, Canada, pp. 492–497.

Hua

Zhang

Guan

(2017) Distributed adaptive neural network output tracking of leader-following high-order stochastic nonlinear multiagent systems with unknown dead-zone input. IEEE Transactions on Cybernetics 47: 177–185.

Hua

Ding

(2011) Model following controller design for large-scale systems with time-delay interconnections and multiple dead-zone inputs. IEEE Transactions on Automatic Control 56: 962–968.

Hull

(2003) Optimal Control Theory for Applications. New York: Springer.

Pan

Zhao

(2022) Adaptive fault-tolerant optimized formation control for perturbed nonlinear multiagent systems. International Journal of Robust and Nonlinear Control 32: 3386–3407.

Song

Wen

(2023) Reinforcement learning based optimized sliding-mode consensus control of high-order nonlinear canonical dynamic multiagent system. IEEE Systems Journal 17: 6302–6311.

(2020) A novel dead zone reaching law of discrete-time sliding mode control with disturbance compensation. IEEE Transactions on Industrial Electronics 67: 4815–4825.

10.

Mozer

S, C M

Hasselmo

(2005) Reinforcement learning: An introduction. IEEE Transactions on Neural Networks 16: 285–286.

11.

Ren

Zhao

Zhang

, et al. (2021) Adaptive neural-network boundary control for a flexible manipulator with input constraints and model uncertainties. IEEE Transactions on Cybernetics 51: 4796–4807.

12.

Ren

Zhu

Zhao

, et al. (2022) Adaptive fault-tolerant boundary control for a flexible string with unknown dead zone and actuator fault. IEEE Transactions on Cybernetics 52: 7084–7093.

13.

Shen

Wang

, et al. (2022) Adaptive output-feedback control for a class of nonlinear systems based on optimized backstepping technique. International Journal of Adaptive Control and Signal Processing 36: 1077–1097.

14.

Tong

Sui

(2015) Observed-based adaptive fuzzy tracking control for switched nonlinear systems with dead-zone. IEEE Transactions on Cybernetics 45: 2816–2826.

15.

Vamvoudakis

Lewis

(2010) Online actor critic algorithm to solve the continuous-time infinite horizon optimal control problem. Automatica 46: 878–888.

16.

Wang

Hong

(2004) Robust adaptive control of a class of nonlinear systems with unknown dead-zone. Automatica 40: 407–413.

17.

Wen

Chen

CLP

(2021) Simplified optimized backstepping control for a class of nonlinear strict-feedback systems with unknown dynamic functions. IEEE Transactions on Cybernetics 51: 4567–4580.

18.

Wen

Chen

CLP

, et al. (2019a) Optimized adaptive nonlinear tracking control using actor-critic reinforcement learning strategy. IEEE Transactions on Industrial Informatics 15: 4969–4977.

19.

Wen

Philip Chen

(2020) Simplified optimized control using reinforcement learning algorithm for a class of stochastic nonlinear systems. Information Sciences 517: 230–243.

20.

Wen

Dou

(2023) Adaptive fuzzy leader–follower consensus control using sliding mode mechanism for a class of high-order unknown nonlinear dynamic multi-agent systems. International Journal of Robust and Nonlinear Control 33: 545–558.

21.

Wen

(2018) Optimized backstepping for tracking control of strict-feedback systems. IEEE Transactions on Neural Networks and Learning Systems 29: 3850–3862.

22.

Wen

Chen

CLP

, et al. (2019b) Adaptive tracking control of surface vessel using optimized backstepping technique. IEEE Transactions on Cybernetics 49: 3420–3431.

23.

Wen

Hao

Feng

, et al. (2022) Optimized backstepping tracking control using reinforcement learning for quadrotor unmanned aerial vehicle system. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52: 5004–5015.

24.

Wen

(2022) Optimized leader–follower consensus control using reinforcement learning for a class of second-order nonlinear multiagent systems. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52: 5546–5555.

25.

Wen

Niu

(2022) Optimized tracking control based on reinforcement learning for a class of high-order unknown nonlinear dynamic systems. Information Sciences 606: 368–379.

26.

Wen

Chen

CLP

Liu

, et al. (2015) Neural-network-based adaptive leader-following consensus control for second-order non-linear multi-agent systems. IET Control Theory & Applications 9: 1927–1934.

27.

Yang

Wei

, et al. (2018) Reinforcement learning for robust adaptive control of partially unknown nonlinear systems subject to unmatched uncertainties. Information Sciences 463–464: 307–322.

28.

Yoo

Park

Choi

(2009) Decentralized adaptive stabilization of interconnected nonlinear systems with unknown non-symmetric dead-zone inputs. Automatica 45: 436–443.

29.

Zhou

Wen

Zhang

(2006) Adaptive output control of nonlinear systems with uncertain dead-zone nonlinearity. IEEE Transactions on Automatic Control 51: 504–511.