Adaptive periodic event-triggered control for missile-target interception system with finite-horizon convergence

Abstract

In this paper, the optimal control problem for finite-time missile-target interception systems is posed in a finite-horizon two-player zero-sum (ZS) differential game framework using a periodic event-triggered (PET) scheme. To solve the optimal control problem, a time-varying Hamilton-Jacobi-Issac (HJI) equation and a time-dependent cost function are constructed to deal with finite-horizon constraints, and an event-based periodic adaptive dynamic programming (ADP) algorithm is employed to find the Nash equilibrium solution for the designed HJI equation. Comparing with the traditional continuous event-triggered (ET) scheme, the proposed PET scheme only verifies the event-triggered conditions at periodic sampling instants, which reduces resource consumption in monitoring and excludes the Zeno behavior. A single critic neural network (CNN) is used to implement the proposed event-based optimal control algorithm, which reduces approximate errors bust also simplifies structures. Further, an additional error term is added in the designed weight updating law to such that the terminal constraint is also minimized over time. By resorting to Lyapunov function approach, some sufficient conditions are derived to achieve the uniformly ultimately bounded (UUB) of the ET closed-loop system and the estimation weight error of CNN. Finally, a missile-target interception system is introduced to illustrate the efficiency of the presented methods.

Keywords

Missile-target interception two-player ZS differential game finite-horizon adaptive dynamic programming periodic event-triggered strategy

Introduction

In the field of intercept guidance, modern guidance laws have received extensive attention due to extremely fast development of modern control theory and computer technology. There are numerous reported on guidance laws for missile-target interception system, such as the optimal guidance law (OGL) (He and Lee, 2018), the differential game guidance law (DGL) (Sun et al., 2016), the sliding mode guidance law (SMGL) (Guo et al., 2019; Wang, 2017) and the robust guidance law (NGL) (Li and Ji, 2016; Song and Song, 2019). The optimal control theory is firstly applied to the study of guidance law in Kishi and Bettwy (1965), after which plenty of theoretical results of OGL have also been provided in recent years. However, the OGL requires real-time estimation of remaining flight time and total motion information of the target, which inevitably degrades the guidance precision when existing a large remaining time estimation error. To overcome this drawback, differential games are put forward by Rufus (1954), which combines modern optimal control with game theory by using the dynamic programming method. At present, differential games have been widely employed in seeking the optimal strategy of both missiles and targets (Perelman et al., 2011; Shima and Golan, 2007). Note that these researches mentioned above failed to consider the optimal control problem for nonlinear systems, especially for nonlinear differential games.

In the terminal guidance interception problem of the maneuvering target, the target tries to maximize the miss distance while the interceptor seeks the optimal strategy to minimize the miss distance. Therefore, the problem of missile-target interception can be construct as a optimal control problem of nonlinear two-player zero-sum (ZS) differential games. However, it is almost impossible to seek a Nash equilibrium solution of the corresponding Hamilton-Jacobi-Isaac (HJI) equation in case of nonlinear differential games. In this circumstance, the ADP method is presented to approximate the optimal solution for the designed HJI equation by using neural networks (NNs). On account of ADP is valid in solving optimal control problem, plenty of theoretical results are obtained in the field of multi-error constraint control, fault-tolerant control, power system stability control and optimal battery energy control (Dai et al., 2019; Liu et al., 2018a; Tang et al., 2014; Wei et al., 2017). The authors adopt an actor-critic structure to approximate the optimal function and the optimal control strategy in Dierks (2012) and White et al. (1994). For uncertain nonlinear systems, an ADP-based method is employed to solve the Hamilton-Jacobi-Bellman (HJB) equation forward-in-time using a three-NN-structure. Based on ADP technique (Bhasin et al., 2013), the robust optimal control problem for affine nonlinear systems are considered in Ding et al. (2014), and the full state constraints systems are discussed in Gao et al. (2018) and Liu et al. (2018b, 2019). Further, massive of significant results for ZS differential games and NZS differential games have been reported in Vamvoudakis and Lewi, (2011), Zhang et al. (2017a), Sun and Liu (2018b) and Qu et al. (2018) via NNs. However, finite-horizon constraints in ADP design is not discussed in these researches mentioned above, even if finite-horizon convergence has been substantiated to have better robustness and anti-disturbance ability, which urges us to study it further.

The finite-horizon optimal control problem for nonlinear systems is more complicated than under circumstance of the infinite-horizon. The difficulties result from the time-varying HJI equation and the time-to-go dependent cost function, which are time-invariant under infinite-horizon. In addition, the finite-horizon cost function has to satisfy a terminal constraint condition, which is taken as zero for infinite-horizon. To address the above issues, time-varying activation functions and an additional error term are employed to approximate the coupled HJB equation while guaranteeing that the terminal constraint error is minimized in Zhao et al. (2015). In Wang et al. (2011, 2012), the optimal tracking problem for continuous systems and discrete systems are considered, where a iterative ADP algorithm be used to design the finite-horizon stabilization controller. However, the actor-critic structure employed in Zhao et al. (2015) and Wang et al. (2011, 2012) will generate a mass of approximate errors. To tackle this issue, the authors in Heydari and Balakrishnan (2013) eliminated the actor NN, and a single critic neuron network (CNN) is used to approximate the time-varying solution of the coupled HJI equation. For uncertain nonlinear systems, Huang (2016) developed a neuro-observer based finite-horizon optimal policy by using an observer-critic structure. Finite-horizon ZS and NZS differential games problem are considered in Cui et al. (2016) and Sun and Liu (2018a), the Nash equilibrium of coupled HJI equations are approximated by a single CNN. By now, most of existing literatures on optimal control problem are mainly taken for a time-driven scheme, with few on event-triggered schemes.

It is worth noting that communication resources and network bandwidth are limited in the actual missile guidance control system. However, most existing works on finite-horizon optimal control use a time-triggered mechanism. This mechanism will increase communication pressure, but also affect the effectiveness of intercepting target by missile. Therefore, the event-triggered ADP algorithm is proposed in Vamvoudakis (2014) to overcome the limitation of the time-triggered scheme. Subsequently, Zhong et al. (2015) and Sahoo et al. (2017) employed an adaptive ET control strategy to settle the optimal control issue of the continuous nonlinear systems. The optimal control problem is further studied in Zhu et al. (2017), when the continuous nonlinear system exist locally unknown dynamics and input constraints. In Wang et al. (2016) and Mu and Wang (2018), the problem of $H_{\infty}$ control for continuous-time nonlinear systems is converted to a optimal control problem for ZS differential game systems, in which the adaptive evaluation learning method is used to attain the event-based optimal control strategy and the time-based optimal perturbation strategy. But until now, the use of ET scheme in ZS differential game systems with finite-horizon constraints is not common, especially under the background of missile target interception. In addition, how to evade the Zeno behavior under the ET manner is also a serious problem to be solved urgently.

On the background of intercepting the maneuvering target, this paper first attempt to investigate the optimal control problem for missile-target interception systems with finite-horizon constraints using a periodic event-triggered adaptive dynamics programming (PETADP) method. At first, the finite-horizon convergence problem for missile-target interception systems is converted to a optimal control problem of two-player ZS differential game with a time-dependent cost function. To save communication costs and computing resources, a PETADP scheme is employed to address the optimal control problem for a class of nonlinear system efficiently. Then, the triggered condition is obtained to ensure that the nonlinear system with finite-horizon constraints is UUB under the event-based optimal controller. Finally, a single CNN is used to approximate the saddle point of the event-based HJI equation online. The main contributions are threefold:

Compared with some existing literatures (Sahoo et al., 2017; Vamvoudakis, 2014; Wang et al., 2016; Zhong et al., 2015; Zhu et al., 2017), this paper proposed PET scheme verified the event-triggered condition only at a fixed periodic, where the minimum inter-event time is greater than or equal to validity period. Obviously, this scheme combines the advantages of both traditional ET schemes and the periodic sampling, the benefits of saved communication resources and computing costs are preserved while the Zeno behavior is avoid.

This article first attempt to dealt with the issues of finite-horizon converge for missile-target interception systems via a PET scheme. Different from the infinite-horizon circumstance (Heydari and Balakrishnan, 2013; Wang et al., 2011, 2012), the finite-horizon HJI equation is time-varying, and the finite-horizon cost function should satisfy with a terminal constraint that is considered as zero under infinite-horizon scenario, which is more complicated. Then, an ET tuning law is proposed for CNN weight vectors, in which the condition of initial stabilising control is relaxed.

In terms of the Lyapunov function approach, some stability criteria with the PET method are derived for the closed-loop nonlinear system and the estimation error vector of CNN weight.

The remainder of this paper is organized as follows. In Section 2, a brief description of the optimal control problem for finite-horizon ZS differential games is proposed, where a periodic event-triggered ADP method is provided to improve communication resource utilization efficiency. A single CNN is designed in section 3 to solve the time-varying HJI equation approximately. Then, the event-triggered closed-loop nominal system and the weight error are guaranteed to be UUB by constructing a appropriate tuning law in Section 4. In Section 5, a missile-target interception system model is introduced to validate the effectiveness of this paper proposed PET optima control scheme. Finally, concluding remarks are given in Section 6, and some standard notations will be used to simplify the upcoming description (see Tables 1–2).

Table 1.

Some related notations.

Notations	Meaning	Notations	Meaning
$R$	the set of real numbers	$τ_{s}$	the minimum sampling periodic based on PEC scheme
$Z$	the set of natural numbers	h	the sampling periodic based on time-triggered scheme
$R^{n}$	the n-dimensional Euclidean space	$λ_{min} (\cdot)$	the minimal eigenvalue of a matrix
$R^{n x m}$	the set of real nXm matrices	$λ_{max} (\cdot)$	the maximal eigenvalue of a matrix
$R^{n x m}$	all positive definite matrices	T	the transpose operation

Problem descriptions

Consider the continuous-time nonlinear two-player ZS differential game as follows

\overset{\cdot}{x} (t) = f (x) + g (x) u (t) + k (x) v (t)

(1)

where $x (t) \in R^{n}$ is the state vector, $u (t) \in R^{m}$ and $v (t) \in R^{s}$ are two input vectors for two players, respectively. $g (x) \in R^{n \times m}$ , $f (x) \in R^{n}$ and $k (x) \in R^{n \times s}$ are differentiable nonlinear dynamics with $f (0) = 0$ . It is clear that, $x = 0$ is an equilibrium point of system (1). Now, Assumption 1 and Assumption 2 are provided for system (1).

Assumption 1: The nonlinear dynamics $f (\cdot)$ , $g (\cdot)$ and $k (\cdot)$ are locally Lipschitz continuous on a compact set $Ω \in R^{n}$ , and the system (1) is controllable.

Assumption 2: $g (\cdot)$ and $k (\cdot)$ are bounded by constants $g_{\max}$ and $k_{\max}$ , that is, $∥ g (\cdot) ∥ ⩽ g_{\max}, ∥ k (\cdot) ∥ ⩽ k_{\max} .$

Remark 1: As we all know, external disturbances and control inputs for many practical nonlinear systems are always energy bounded signals. Therefore, the constraint conditions in Assumptions 1–2 are reasonable, and they are widely used in existing literatures (Fu and Chai, 2017; Sun and Liu, 2018c).

Our aim is to propose a controller $u (t)$ to minimize the time-dependent cost function, meanwhile, construct anther controller $v (t)$ to maximize the cost function as follows

{\begin{matrix} V (x, t_{0}) = ϕ (x (t_{f}), t_{f}) + \int_{t_{0}}^{t_{f}} r (x, u, v) dt, \\ V (x, t_{f}) = ϕ (x (t_{f}), t_{f}) . \end{matrix}

(2)

Defining the utility function $r (x, u, v) = Q (x, t) + u^{T} R_{1} u - v^{T} R_{2} v$ , where $Q (x, t) > 0$ is a positive definite function and $R_{1} > 0, R_{2} > 0$ are matrices with appropriate dimension. In terms of Cholesky decomposition, we can obtain $R_{1} = r_{1}^{T} r_{1}$ , $R_{2} = r_{2}^{T} r_{2}$ , where $r_{1}$ and $r_{2}$ are proper lower triangular matrices.

Remark 2: $ϕ (x (t_{f}), t_{f})$ is introduced to deal with finite-horizon constraints, and $ϕ (x (t_{f}), t_{f})$ is called a terminal cost, $t_{f}$ denotes the fixed final time. Different from the infinite-horizon scenario (Cavalieri et al., 2013; Jagat and Sinclair, 2017; Parras et al., 2016), the finite-horizon cost function $V (x, t)$ is time-dependent.

Suppose that $V (x, t) \in C^{1}$ , the infinitesimal equivalent of (2) is defined by

V_{x}^{T} [f (x) + g (x) u (t) + k (x) v (t)] + r (x, u, v) + V_{t} = 0,

(3)

where $V_{x} = \frac{\partial V (x, t)}{\partial x}$ , $V_{t} = \frac{\partial V (x, t)}{\partial t}$ .

Remark 3: Clearly $V (x, t)$ is a time-dependent solution to the time-varying partial differential equation (3). Based on the given control pair $(u (t), v (t))$ and the backward inference method, $V (x, t)$ can be obtained from fixed terminal time $t_{f}$ and terminal cost function $V (x, t_{f})$ . Note that the term $V_{t}$ is generated by the time-dependent cost function $V (x, t)$ while does not appear in the infinite-horizon scenario.

Defining the Hamiltonian function including explicitly time for system (1) as

\begin{matrix} H (x, u, v) = V_{x}^{T} [f (x) + g (x) u (t) + k (x) v (t)] \\ + V_{t} + Q (x, t) + u^{T} R_{1} u - v^{T} R_{2} v . \end{matrix}

(4)

The optimal cost function of system (1) is derived

{\begin{matrix} V^{*} (x, t) = min_{u} max_{v} V (x, t) = max_{v} min_{u} V (x, t), \\ V^{*} (x, t_{f}) = ϕ (x (t_{f}), t_{f}) . \end{matrix}

(5)

In accordance with the stationary conditions, the optimal control strategies can be calculated by

\frac{\partial H (x, u, v)}{\partial u} = 0 \Rightarrow u^{*} (x, t) = - \frac{1}{2} R_{1}^{- 1} g^{T} (x) V_{x}^{*},

(6a)

\frac{\partial H (x, u, v)}{\partial v} = 0 \Rightarrow v^{*} (x, t) = \frac{1}{2} R_{2}^{- 1} k^{T} (x) V_{x}^{*} .

(6b)

Substituting (6a) and (6b) into (4), the time-varying HJI equation can be rewritten by

\begin{matrix} H (x, u^{*}, v^{*}) = V_{t}^{*} + (V_{x}^{*})^{T} f (x) + Q (x, t) \\ - \frac{1}{4} (V_{x}^{*})^{T} g (x) R_{1}^{- 1} g^{T} (x) V_{x}^{*} \\ + \frac{1}{4} (V_{x}^{*})^{T} k (x) R_{2}^{- 1} k^{T} (x) V_{x}^{*} = 0 . \end{matrix}

(7)

Periodic event-triggered scheme

In this section, the traditional periodic sampling mechanism is considered firstly, that is, the controller is only update at periodic sampling instants. In general, the sampling period of the system is assumed as a constant $h > 0$ , and the sample sequence is given below

t_{0}, t_{0} + h, t_{0} + 2 h, t_{0} + 3 h, t_{0} + 4 h, \dots, t_{0} + l h, l \in ℕ .

Among them, some changes in sampled data are negligible or have no impact on the performance we are studying, so there is no need to transmit. In this sense, the traditional periodic sampling mechanism will generate too many redundant signals, thus causing a waste of computing resources. To overcome this obstacle, the event-based periodic sampling scheme is applied in this paper, where the periodically sampled signal can be transmitted only when the defined ET condition is satisfied. For convenience and without loss of generality, the initial time is supposed to zero, then the framework of the periodic event-triggered sampling is given

Remark 4: In contrast to references (Sahoo et al., 2017; Vamvoudakis, 2014; Wang et al., 2016; Zhong et al., 2015; Zhu et al., 2017), the ET condition only needs to be verified at a fixed periodic under this article proposed the PET scheme. In Figure 1, the minimum inter-event time $τ_{s}$ is greater than or equal to the sampling period for the time-triggered scheme $h$ . Obviously, the computing costs of the detection process is further reduced and the Zeno behavior is avoid.

Figure 1.

Illustration of periodic event-triggered sampling.

As show in Figure 1, we first consider the periodic sampling scheme, then the event-triggered scheme is introduced to enhance the utilization rate of limited communications resource. Under the PETS, only periodic sampling states $x (lh)$ that satisfy with the defined event-triggered conditions can be transmitted, and the event-triggered sampled states are denoted as

x (t_{r} h) \overset{Δ}{=} x_{r} (t) \forall t \in [t_{r} h, t_{r + 1} h), r \in N,

where $t_{r} h$ denote event-triggered instants. Define the event-based threshold error between $x (lh)$ and $x (t_{r} h)$ as

\begin{matrix} e (t_{r} h) \overset{Δ}{=} e_{r} (t) = x (t_{r} h) - x ((t_{r} + l) h) \forall t \in [t_{r} h, t_{r + 1} h), \\ l \in N, r \in N, \end{matrix}

where $t_{r + 1} h = t_{r} h + min_{l} {lh | e_{r}^{T} (t) Φ e_{r} (t) > ∥ e_{M} ∥^{2}}$ , $e_{M}$ is the unknown event-triggered condition. And $Φ > 0$ denotes the coefficient of the threshold, which can affect the number of event-triggered.

As the PET scheme is introduced, the state feedback controller $u (x (t_{r} h)) \overset{Δ}{=} μ ({\hat{x}}_{r})$ is update only at the event-triggered instants. Note that $μ ({\hat{x}}_{r})$ is a piecewise constant function with states ${\hat{x}}_{r}$ as independent variables and continuous from the right everywhere. Under the PET scheme, the system (1) and state feedback control law (6) is reconstructed as

\begin{matrix} \overset{\cdot}{x} (t) = f (x (t)) + g (x (t)) μ (e_{r} (t) + x ((t_{r} + l) h)) + k (x (t)) v (t), \\ \forall t \in [t_{r} h, t_{r + 1} h), r \in N, \end{matrix}

(8)

μ^{*} ({\hat{x}}_{r}) = - \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) V^{*} ({\hat{x}}_{r}) \forall t \in [t_{r} h, t_{r + 1} h), r \in N .

(9)

This together with the Hamiltonian function (4) and the time-triggered disturbance stategy (7), then, the event-based HJI equation is reconstructed as

\begin{matrix} H (x, μ^{*} ({\hat{x}}_{r}), v (x), V_{x}^{*} (x)) \\ = Q (x, t) + V_{t}^{*} + (μ^{*} ({\hat{x}}_{r}))^{T} R_{1} μ^{*} ({\hat{x}}_{r}) - v^{T} (t) R_{2} v (t) \\ + V_{x}^{T} [f (x) + g (x) μ ({\hat{x}}_{r}) + k (x) v (t)] . \end{matrix}

(10)

In order to analyze conveniently, the following Assumption is introduced which is reasonable and easy to implement in the event-based control systems. The assumption is widely used in literatures (Wang et al., 2016; Vamvoudakis, 2014; Zhang et al., 2017a).

Assumption 3: The optimal controller $u^{*} (\cdot)$ is Lipschitz continuous with respect to $e_{r} (t)$ , that is

∥ u^{*} (x (t)) - μ^{*} ({\hat{x}}_{r}) ∥ ⩽ L ∥ e_{r} (t) ∥,

(11)

where $L$ is a constant and $L > 0$ .

Lemma 1: Consider time-triggered HJI equation (7) and event-triggered HJI equation (10), if Assumption 1 holds, then the following relationship is satisfied

\begin{matrix} ∥ H (x, u^{*} (x), v^{*} (x), V_{x}^{*} (x)) - H (x, μ^{*} ({\hat{x}}_{r}), v^{*} (x), V_{x}^{*} (x)) \\ ∥ ⩽ L^{2} ∥ r_{1} ∥^{2} ∥ e_{r} (t) ∥^{2} . \end{matrix}

(12)

Proof: Based on (7) and (10), one has

\begin{matrix} H (x, u^{*}, v^{*}, V_{x}^{*}) - H (x, μ^{*} ({\hat{x}}_{r}), v^{*}, V_{x}^{*}) \\ = (u^{*} (x))^{T} R_{1} u^{*} (x) - (μ^{*} ({\hat{x}}_{r}))^{T} R_{1} μ^{*} ({\hat{x}}_{r}) \\ + (u^{*} (x))^{T} g^{T} (x) V_{x}^{*} - (μ^{*} ({\hat{x}}_{r}))^{T} g^{T} (x) V_{x}^{*} \\ = - (u^{*} (x) - μ^{*} ({\hat{x}}_{r}))^{T} R_{1} (u^{*} (x) - μ^{*} ({\hat{x}}_{r})) . \end{matrix}

According to Assumption 3, the inequality (13) is obtained.

Theorem 1: For the nonlinear two-player ZS differential game (1) with the finite-horizon cost function (2), suppose Assumptions 1–3 all hold. The time-triggered disturbance law and the event-trigged control law are given by (6b) and (9), respectively, for all $t \in [t_{r} h, t_{r + 1} h), r \in N$ , then the event-triggered closed-loop system (8) is asymptotic stable, if the following event-triggered condition holds

\begin{matrix} ∥ e_{r} (t) ∥^{2} ⩽ \frac{(1 - η_{1}^{2}) λ_{\min} (q) ∥ x ∥^{2} + ∥ r_{1}^{T} μ^{*} ({\hat{x}}_{r}) ∥^{2} - ∥ r_{2}^{T} v^{*} (x) ∥^{2}}{L^{2} ∥ r_{1} ∥^{2}} \\ \overset{Δ}{=} ∥ e_{M} ∥^{2} . \end{matrix}

(13)

where $e_{M}$ is the event-triggered threshold and $η_{1} \in (0, 1)$ is the designed parameter of the sample frequency.

Proof: The optimal value function $V^{*} (x, t)$ can be regarded as a Lyapunov function, since it is a time-varying solution to the time-varying HJI equation. Taking the time derivation of $V^{*} (x, t)$ yields

{\overset{\cdot}{V}}^{*} (x (t), t) = (V_{x}^{*})^{T} [f (x) + g (x) μ^{*} ({\hat{x}}_{r}) + k (x) v^{*} (x)] + V_{t}^{*} .

(14)

Combining (6a) and (6b), yields

{\begin{matrix} {(V_{x}^{*})}^{T} g (x) = - 2 (u^{*} (x {))}^{T} R_{1}, \\ {(V_{x}^{*})}^{T} k (x) = 2 (v^{*} (x {))}^{T} R_{2} . \end{matrix}

(15)

Since $Q (x, t) > 0$ , there is a matrix $q > 0$ , such that $x^{T} qx < Q (x, t)$ . Then combining with (7), (15) and (14), the time derivation of $V^{*} (x (t), t)$ becomes

\begin{matrix} {\overset{\cdot}{V}}^{*} (x (t), t) = \frac{1}{4} (V_{x}^{*})^{T} g (x) R_{1}^{- 1} g^{T} (x) V_{x}^{*} - \frac{1}{4} (V_{x}^{*})^{T} k (x) R_{2}^{- 1} k^{T} (x) V_{x}^{*} \\ - 2 (u^{*} (x))^{T} R_{1} μ^{*} ({\hat{x}}_{r}) + 2 (v^{*} (x))^{T} R_{2} v^{*} (x) - Q (x, t), \\ ⩽ L^{2} ∥ r_{1} ∥^{2} ∥ e_{r} (t) ∥^{2} + (η_{1}^{2} - 1) λ_{\min} (q) ∥ x ∥^{2} \\ - ∥ r_{1} μ^{*} ({\hat{x}}_{r}) ∥^{2} + ∥ r_{2} v^{*} (x) ∥^{2} - η_{1}^{2} λ_{\min} (q) ∥ x ∥^{2} . \end{matrix}

Therefore, $\overset{\cdot}{V} (x (t), t) ⩽ - η_{1}^{2} λ_{\min} (q) ∥ x ∥^{2} < 0, \forall x \neq 0$ , as long as the inequality (13) is satisfied.

Finite-horizon neural networked implementation using the event-triggered adaptive critic controller

In this section, the single CNN is applied to approximate the time-varying cost function $V (x (t), t)$ and the terminal constraint as follows

V (x, t) = W_{c}^{T} σ_{c} (x, t_{f} - t) + ε_{c} (x, t),

(16a)

V (x, t_{f}) = W_{c}^{T} σ_{c} (x (t_{f}), 0) + ε_{c} (x, t_{f}),

(16b)

where $W_{c} \in R^{l_{c}}$ is the ideal NN weight to be calculated, and $l_{c} \in Z, l_{c} > 0$ represents the number of hidden-layer neurons. $σ_{c} (\cdot) = [σ_{c 1} (\cdot), σ_{c 2} (\cdot), \dots, σ_{c l_{c}} (\cdot)] \in R^{l_{c}}$ is the time-dependent activation function, and $ε_{c} (\cdot) \in R$ is the NN function reconstruction error. As in Sun and Liu (2018b) and Qu et al. (2018), the activation function $σ_{c} (\cdot)$ is bounded.

It follows from (16a) that

V_{x} = \nabla_{x}^{T} σ_{c} (x, t_{f} - t) W_{c} + \nabla_{x} ε_{c} (x, t),

(17a)

V_{t} = \nabla_{t}^{T} σ_{c} (x, t_{f} - t) W_{c} + \nabla_{t} ε_{c} (x, t),

(17b)

where

\nabla_{x} σ_{c} (x, t_{f} - t) = \frac{\partial σ_{c} (x, t_{f} - t)}{\partial x}, \nabla_{x} ε_{c} (x, t) = \frac{\partial ε_{c} (x, t)}{\partial x},

\nabla_{t} σ_{c} (x, t_{f} - t) = \frac{\partial σ_{c} (x, t_{f} - t)}{\partial t}, \nabla_{t} ε_{c} (x, t) = \frac{\partial ε_{c} (x, t)}{\partial t} .

On account of the ideal weight is unknown, the finite-horizon cost function and the terminal constraint are approximated by a critic NN

\hat{V} (x, t) = {\hat{W}}_{c}^{T} σ_{c} (x, t_{f} - t),

(18a)

\hat{V} (x (t_{f}), t_{f}) = {\hat{W}}_{c}^{T} σ_{c} (\hat{x} (t_{f}), 0),

(18b)

where $\hat{V} (x, t)$ denotes the approximate cost function, and the approximate cost function at the terminal time is denoted by $\hat{V} (x (t_{f}), t_{f})$ . $σ_{c} (\hat{x} (t_{f}), 0) \in R^{l_{c}}$ represents the activation function, where the estimated state $\hat{x} (t_{f})$ can be calculated by the current state and the system dynamics. Further, the partial derivative of (18a) and (18b) with respect to $x$ and $t$ are obtained respectively

{\begin{matrix} {\hat{V}}_{x} = \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c}, \\ {\hat{V}}_{t} = \nabla_{t}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c}, \end{matrix}

(19)

From (6a), (9), (17a) and (17b), the event-triggered control strategy and the time-based disturbance strategy can be described by

μ ({\hat{x}}_{r}) = - \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{f} - t) W_{c} + ε_{u}^{*},

(20a)

v (x) = \frac{1}{2} R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) W_{c} + ε_{v}^{*},

(20b)

where $ε_{u}^{*} = - \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x} ε_{c} ({\hat{x}}_{r}, t)$ , $ε_{v}^{*} = \frac{1}{2} R_{2}^{- 1} k^{T} (x) \nabla_{x} ε_{c} (x, t)$ .

Combining (16a) with (16b) gives

\begin{matrix} H (x, μ ({\hat{x}}_{r}), v (x), W_{c}) = Q (x, t) \\ + μ^{T} ({\hat{x}}_{r}) R_{1} μ ({\hat{x}}_{r}) - v^{T} (x) R_{2} v (x) + W_{c}^{T} \nabla_{t} σ_{c} (x, t_{f} - t) \\ + W_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) [f (x) + g (x) μ ({\hat{x}}_{r}) + k (x) v (x)], \\ \overset{Δ}{=} ε_{HJI} . \end{matrix}

(21)

Here, $ε_{HJI} = - \nabla_{t}^{T} ε (x, t) - \nabla_{x}^{T} ε (x, t) [f (x) + g (x) μ ({\hat{x}}_{r}) + k (x) v (x)]$ represents the reconstruction error of CNN approximation. Further, according to (19), it follows

\hat{μ} ({\hat{x}}_{r}) = - \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{f} - t) {\hat{W}}_{c},

(22a)

\hat{v} (x) = \frac{1}{2} R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c} .

(22b)

Noticing (18a) and (18b), the approximate event-triggered HJI equation is described by

\begin{matrix} \hat{H} (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) = Q (x, t) \\ + {\hat{μ}}^{T} ({\hat{x}}_{r}) R_{1} \hat{μ} ({\hat{x}}_{r}) - {\hat{v}}^{T} (x) R_{2} \hat{v} (x) + {\hat{W}}_{c}^{T} \nabla_{t} σ_{c} (x, t_{f} - t) \\ + {\hat{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) [f (x) + g (x) \hat{μ} ({\hat{x}}_{r}) + k (x) \hat{v} (x)], \\ \overset{Δ}{=} e_{c} . \end{matrix}

(23)

In order to settle the issue of finite-horizon constraints in the ZS differential game systems, the time-varying cost function and its terminal constraint should be considered. Therefor, the terminal constraint estimation error is introduced according to (18b)

\begin{matrix} e_{t_{f}} = V (x (t_{f}), t_{f}) - \hat{V} (x (t_{f}), t_{f}), \\ = {\tilde{W}}_{c}^{T} σ_{c} (\hat{x} (t_{f}), 0) + W_{c}^{T} {\tilde{σ}}_{c} (x (t_{f}), 0) + ε (t_{f}), \end{matrix}

(24)

where ${\tilde{σ}}_{c} (x (t_{f}), 0) = σ_{c} (x (t_{f}), 0) - σ_{c} (\hat{x} (t_{f}), 0)$ .

Lemma 2: (Yang et al., 2013) For system (1) with finite-horizon cost function (2), the control strategy (22a) and (22b). Assume $J (x)$ is a Lyapunov function, which is continuously differentiable and radially unbounded, and satisfy with the following inequality

\overset{\cdot}{J} (x) = J_{x}^{T} [f (x) + g (x) u^{*} (x) + k (x) v^{*} (x)] < 0 .

If there exist a positive definite function $Λ (x) \in R^{n \times n}$ , $lim_{x \to \infty} Λ (x) = \infty$ , then the following inequality holds

J_{x}^{T} (f (x) + g (x) u^{*} (x) + k (x) v^{*} (x)) < - J_{x}^{T} Λ (x) J_{x},

(25)

where $J_{x} = \frac{\partial J (x)}{\partial x}$ .

Combing with the control input (22a) and (22b), the derivative of Lyapunov function $J (x)$ for system (1) can be calculated by

\begin{matrix} θ \overset{Δ}{=} J_{x}^{T} (f (x) + g (x) u^{*} (x) + k (x) v^{*} (x)), \\ = J_{x}^{T} f (x) - \frac{1}{2} J_{x}^{T} g (x) R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} (\hat{x}, t_{f} - t) {\hat{W}}_{c} \\ + \frac{1}{2} J_{x}^{T} k (x) R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c} . \end{matrix}

In terms of the gradient descent method, we develop the following update law for $θ$ ,

\begin{matrix} - \frac{\partial θ}{\partial {\hat{W}}_{c}} = \frac{- 1}{\partial {\hat{W}}_{c}} \cdot \partial [J_{x}^{T} f (x) - \frac{1}{2} J_{x}^{T} g (x) R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} (\hat{x}, t_{f} - t) {\hat{W}}_{c} \\ + \frac{1}{2} J_{x}^{T} k (x) R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c}], \\ = \frac{1}{2} \nabla_{x} σ_{c} (\hat{x}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} (x) J_{x} \\ - \frac{1}{2} \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) J_{x} . \end{matrix}

In order to obtain ${\hat{W}}_{c}$ to minimize the Hamiltonian function that satisfied with the terminal constraint during the learning process of the CNN, the total squared error is defined as

E_{total} = \frac{1}{2} e_{c}^{T} e_{c} + \frac{1}{2} e_{t_{f}}^{T} e_{t_{f}} .

(26)

Generally, the optimal control strategy and the disturbance strategy are not available in practical engineering applications. Therefore, approximate control strategy (22a) and approximate disturbance strategy (22b) are applied to the learning process in the neural network. According to the normalized gradient descent algorithm (Liu et al., 2015; Sun and Liu, 2018b,c), the event-triggered weight vector ${\hat{W}}_{c}$ is tuned as follows

\begin{matrix} {\overset{\cdot}{\hat{W}}}_{c} = - α_{c} \frac{1}{{(1 - β_{1}^{T} β_{1})}^{2}} \frac{\partial (\frac{1}{2} e_{c}^{T} e_{c})}{\partial {\hat{W}}_{c}} - α_{c} \frac{1}{{(1 - β_{2}^{T} β_{2})}^{2}} \frac{\partial (\frac{1}{2} e_{t_{f}}^{T} e_{t_{f}})}{\partial {\hat{W}}_{c}}, \\ = - α_{c} \frac{β_{1}}{{(1 - β_{1}^{T} β_{1})}^{2}} (r (x, \hat{μ} ({\hat{x}}_{r}), v (x)) + β_{1}^{T} {\hat{W}}_{c}) \\ - α_{c} \frac{β_{2}}{{(1 - β_{2}^{T} β_{2})}^{2}} ({\tilde{W}}_{c} σ_{c} (\hat{x} (t_{f}), 0) + W_{c}^{T} {\tilde{σ}}_{c} (x (t_{f}), 0) + ε (t_{f})) \\ - \frac{1}{2} α_{c} Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) J_{x} \\ + \frac{1}{2} α_{c} Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} (x) J_{x}, \end{matrix}

(27)

where $β_{1} = \nabla_{t} σ_{c} (x, t_{f} - t) + \nabla_{x} σ_{c} (x, t_{f} - t) (f (x) + g (x) {\hat{μ}}^{T} ({\hat{x}}_{r}) + k (x) {\hat{v}}^{T} (x))$ and $β_{2} = - σ_{c} (\hat{x} (t_{f}), 0)$ . $Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x))$ is defined as

\begin{matrix} Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) = \\ {\begin{matrix} 0, \overset{\cdot}{J} (x) = J_{x}^{T} (f (x) + g (x) \hat{μ} ({\hat{x}}_{r}) + k (x) \hat{v} (x)) < 0 \\ 1, otherwise \end{matrix} \end{matrix}

(28)

Defining ${\tilde{W}}_{c} = W_{c} - {\hat{W}}_{c}$ , and combining (21), (23) and (24), the time derivative of ${\tilde{W}}_{c}$ can be written as

\begin{matrix} {\overset{\cdot}{\tilde{W}}}_{c} = - {\overset{\cdot}{\hat{W}}}_{c} = α_{c} \frac{β_{1}}{{(1 - β_{1}^{T} β_{1})}^{2}} (- {\tilde{W}}_{c}^{T} β_{1} + ε_{HJI}) \\ + α_{c} \frac{β_{2}}{{(1 - β_{2}^{T} β_{2})}^{2}} ({\tilde{W}}_{c} σ_{c} (\hat{x} (t_{f}), 0) + W_{c}^{T} {\tilde{σ}}_{c} (x (t_{f}), 0) + ε (t_{f})) \\ - \frac{1}{2} α_{c} Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} (x) J_{x} \\ + \frac{1}{2} α_{c} Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) J_{x} . \end{matrix}

(29)

Remark 5: In fact, the first term and the second term in (29) are employed to minimize the approximate event-triggered Hamiltonian function and the event-triggered terminal constraint estimation error, respectively.

From the definition (28), it is clear that the third term and the fourth term are introduced to such that the system states are bounded during the learning phase of the neural network. These two terms are zero when the nonlinear system is stable, and that will be activated when an unstable signal is generated.

Stability analysis

Now, some assumptions are given, which can usually stated in the ADP literatures, for instance (Liu et al., 2009; Sahoo et al., 2015; Wang et al., 2017; Zhang et al., 2017b).

Assumption 4: The control dynamic $g (x)$ is Lipschitz continuous on a compact set $Ω \in R^{n}$ , and satisfy with $∥ g (x) - g ({\hat{x}}_{r}) ∥ ⩽ A ∥ e_{r} (t) ∥$ , where $A > 0$ is a constant.

Assumption 5: Let $\nabla_{σ_{c_{\max}}}$ , $\nabla_{ε_{c_{\max}}}$ and $ε_{HJ I_{\max}}$ be positive constants

(1) The partial derivative of activation function $σ_{c} (\cdot)$ is Lipschitz continuous on a compact set $Ω \in R^{n}$ , and satisfy with $∥ \nabla_{σ_{c}} (x) - \nabla_{σ_{c}} ({\hat{x}}_{r}) ∥ ⩽ B ∥ e_{r} (t) ∥$ , where $B > 0$ is a constant

(2) $|| \nabla_{σ_{c}} (x) || ⩽ \nabla_{σ_{c_{\max}}}$ ,

(3) $|| \nabla_{ε_{c}} (x) || ⩽ \nabla_{ε_{c_{\max}}}$ ,

(4) $|| ε_{HJI} || ⩽ ε_{HJ I_{\max}}$ .

Theorem 2: For the nonlinear two-player ZS differential game (1) with the finite-horizon cost function (2), suppose Assumptions 1–5 all hold, if the time-triggered disturbance policy and the event-triggered control policy are designed as (22b) and (22a), respectively, the CNN tuning law is set as (29). Then, the event-triggered closed-feedback system and the estimation error of weight vector ${\tilde{W}}_{c}$ are UUB, if the following conditions hold

∥ ξ ∥ > \sqrt{\frac{λ_{1} + λ_{3}}{λ_{\min} (E)}} + \frac{∥ N ∥}{2 λ_{\min} (E)},

(30)

∥ J_{x} ∥ > \sqrt{\frac{λ_{4}}{λ_{\min} (Λ (x))}} + \frac{d_{M}}{2 λ_{\min} (Λ (x))},

(31)

∥ e_{r} (t) ∥ ⩽ \sqrt{\frac{(1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} ∥ R_{1} ∥}{λ_{2} ∥ {\hat{W}}_{c} ∥^{2}}} .

(32)

Proof: Choose the following Lyapunov function candidate

L (x, t) = V^{*} (x, t) + V^{*} ({\hat{x}}_{r}, t) + J^{*} (x) + \frac{1}{2} {\tilde{W}}_{c}^{T} α_{c}^{- 1} {\tilde{W}}_{c},

(33)

where $V^{*} (x)$ denotes the optimal value function for the continuous nonlinear system (1), and $V^{*} ({\hat{x}}_{r})$ represents the optimal value functions for the event-triggered nonlinear system (8).

Case $I$ : Events are not triggered, that is, $t \in [t_{r}, t_{r + 1})$ . From (6a), (6b) and (7), the derivative of $V^{*} (x, t)$ can be described by

\begin{matrix} {\overset{\cdot}{V}}^{*} (x, t) ⩽ - x^{T} qx + (u^{*} (x) - \hat{μ} ({\hat{x}}_{r}))^{T} R_{1} (u^{*} (x) - \hat{μ} ({\hat{x}}_{r})) \\ - (\hat{μ} ({\hat{x}}_{r}))^{T} R_{1} \hat{μ} ({\hat{x}}_{r}) + {\hat{v}}^{T} (x) R_{2} \hat{v} (x) . \end{matrix}

Considering (6a) and (17a), the time-triggered control strategy is derived

u^{*} (x) = - \frac{1}{2} R_{1}^{- 1} g^{T} (x) [\nabla_{x}^{T} σ_{c} (x, t_{f} - t) W_{c} + \nabla_{x} ε_{c} (x, t)] .

(34)

From (22a) and (34), and considering the relationship ${\tilde{W}}_{c} = W_{c} - {\hat{W}}_{c}$ yields

\begin{matrix} {\overset{\cdot}{V}}^{*} (x, t) ⩽ - x^{T} qx + [- \frac{1}{2} (W_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) + \nabla_{x}^{T} ε (x, t)) g (x) \\ + \frac{1}{2} {\hat{W}}_{c}^{T} \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r})] R_{1}^{- 1} \cdot \\ R_{1} [- \frac{1}{2} R_{1}^{- 1} g^{T} (x) (\nabla_{x}^{T} σ_{c} (x, t_{f} - t) W_{c} \\ - \nabla_{x} ε (x, t)) + \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{f} - t) {\hat{W}}_{c}] \\ - \frac{1}{4} {\hat{W}}_{c}^{T} \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{f} - t) {\hat{W}}_{c} \\ + \frac{1}{4} {\hat{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\hat{W}}_{c}, \\ = \frac{1}{4} [- ({\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) + \nabla_{x}^{T} ε (x, t)) g (x) \\ + {\hat{W}}_{c}^{T} (\nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) \\ - \nabla_{x} σ_{c} (x, t_{f} - t) g (x))] R_{1}^{- 1} \\ [- ({\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) + \nabla_{x}^{T} ε (x, t)) g (x) \\ + {\hat{W}}_{c}^{T} (\nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) - \nabla_{x} σ_{c} (x, t_{f} - t) g (x))]^{T} \\ - \frac{1}{4} {\hat{W}}_{c}^{T} K_{1} K_{1}^{T} {\hat{W}}_{c} + \frac{1}{4} {\hat{W}}_{c}^{T} K_{3} K_{3}^{T} {\hat{W}}_{c} - x^{T} qx, \\ ⩽ - x^{T} qx + \frac{1}{2} [({\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) + \nabla_{x}^{T} ε (x, t)) g (x)] R_{1}^{- 1} \cdot \\ [g^{T} (x) (\nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\tilde{W}}_{c} + \nabla_{x} ε (x, t))] \\ + \frac{1}{2} [{\hat{W}}_{c}^{T} (\nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) \\ - \nabla_{x} σ_{c} (x, t_{f} - t) g (x))] R_{1}^{- 1} [(g^{T} ({\hat{x}}_{r}) \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) \\ - g^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t))] {\hat{W}}_{c} - \frac{1}{4} {\hat{W}}_{c}^{T} K_{1} K_{1}^{T} {\hat{W}}_{c} \\ + \frac{1}{4} {\hat{W}}_{c}^{T} K_{3} K_{3}^{T} {\hat{W}}_{c}, \\ ⩽ - x^{T} qx \\ + \frac{1}{2} {\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) g (x) R_{1}^{- 1} g^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) {\tilde{W}}_{c} \\ + {\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) g (x) R_{1}^{- 1} g^{T} (x) \nabla_{x} ε (x, t) \\ - \frac{1}{4} {\hat{W}}_{c}^{T} K_{1} K_{1}^{T} {\hat{W}}_{c} + \frac{1}{4} {\hat{W}}_{c}^{T} K_{3} K_{3}^{T} {\hat{W}}_{c} \\ + \frac{1}{2} \nabla_{x}^{T} ε (x, t) g (x) R_{1}^{- 1} g^{T} (x) \nabla_{x} ε (x, t) \\ + \frac{1}{2} ∥ {\hat{W}}_{c} ∥^{2} ∥ R_{1}^{- 1} ∥ ∥ \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) \\ - \nabla_{x} σ_{c} (x, t_{f} - t) g (x) ∥^{2}, \\ = - x^{T} qx + \frac{1}{2} {\tilde{W}}_{c}^{T} K_{2} K_{2}^{T} {\tilde{W}}_{c} + \frac{1}{2} K_{4} K_{4}^{T} \\ + {\tilde{W}}_{c} K_{2} K_{4}^{T} + \frac{1}{4} {\hat{W}}_{c}^{T} K_{3} K_{3}^{T} {\hat{W}}_{c} - \frac{1}{4} {\hat{W}}_{c}^{T} K_{1} K_{1}^{T} {\hat{W}}_{c} \\ + \frac{1}{2} ∥ {\hat{W}}_{c} ∥^{2} ∥ R_{1}^{- 1} ∥ ∥ \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) \\ - \nabla_{x} σ_{c} (x, t_{f} - t) g (x) ∥^{2}, \end{matrix}

where

\begin{matrix} K_{1} = \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) r_{1}^{- 1}, K_{2} = \nabla_{x} σ_{c} (x, t_{f} - t) g (x) r_{1}^{- 1}, \\ K_{3} = \nabla_{x} σ_{c} (x, t_{f} - t) k (x) r_{2}^{- 1}, K_{4} = \nabla_{x}^{T} ε (x, t) g (x) r_{1}^{- 1} . \end{matrix}

According to Assumptions 4 and 5, one has

\begin{matrix} ∥ \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) - \nabla_{x} σ_{c} (x, t_{f} - t) g (x) ∥^{2} \\ = ∥ \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) + \nabla_{x} σ_{c} (x, t_{f} - t) g ({\hat{x}}_{r}) \\ - \nabla_{x} σ_{c} (x, t_{f} - t) g ({\hat{x}}_{r}) - \nabla_{x} σ_{c} (x, t_{f} - t) g (x) ∥^{2}, \\ ⩽ 2 ∥ (\nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) - \nabla_{x} σ_{c} (x, t_{f} - t)) g ({\hat{x}}_{r}) ∥^{2} \\ + 2 ∥ \nabla_{x} σ_{c} (x, t_{f} - t) (g ({\hat{x}}_{r}) - g (x)) ∥^{2}, \\ ⩽ 2 (B^{2} g_{\max}^{2} + A^{2} \nabla_{x}^{2} σ_{c_{\max}}) ∥ e_{r} (t) ∥^{2} \end{matrix}

Further, the time derivative of Lyapunov function $V^{*} (x, t)$ becomes

\begin{matrix} {\overset{\cdot}{V}}^{*} (x, t) ⩽ - x^{T} qx + \frac{1}{2} {\tilde{W}}_{c}^{T} K_{2} K_{2}^{T} {\tilde{W}}_{c} + \frac{1}{2} K_{4} K_{4}^{T} \\ - \frac{1}{4} {\hat{W}}_{c}^{T} (K_{1} K_{1}^{T} - K_{3} K_{3}^{T}) {\hat{W}}_{c} + {\tilde{W}}_{c} K_{2} K_{4}^{T} \\ + ∥ R_{1}^{- 1} ∥ (B^{2} g_{\max}^{2} + A^{2} \nabla_{x}^{2} σ_{c_{\max}}) ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} . \end{matrix}

In accordance with Lemma 2, the following inequalities hold

\begin{matrix} {\overset{\cdot}{V}}^{*} ({\hat{x}}_{r}, t) = 0, \\ \overset{\cdot}{J} (x) = J_{x}^{T} (f (x) + g (x) \hat{μ} (x_{r}) + k (x) \hat{v} (x)), \\ {\overset{\cdot}{\tilde{W}}}_{c}^{T} α_{c}^{- 1} {\tilde{W}}_{c}, = {\tilde{W}}_{c}^{T} [(\frac{β_{1}}{{(1 - β_{1}^{T} β_{1})}^{2}} (- {\tilde{W}}_{c}^{T} β_{1} + ε_{HJI})) \\ + \frac{β_{2}}{{(1 - β_{2}^{T} β_{2})}^{2}} ({\tilde{W}}_{c}^{T} σ_{c} (\hat{x} (t_{f}), 0) + W_{c}^{T} {\tilde{σ}}_{c} (x (t_{f}), 0) + ε (t_{f})) \\ - \frac{1}{2} Σ (x, \hat{μ} ({\hat{x}}_{r}), v (x)) \nabla_{x} σ_{c} ({\hat{x}}_{r}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} (x) J_{x} \\ + \frac{1}{2} Σ (x, \hat{μ} ({\hat{x}}_{r}), v (x)) \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) J_{x})] . \end{matrix}

(1) $Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) = 0$ , according to the definition of $Σ$ , the term $J_{x}^{T} (f (x) + g (x) \hat{μ} + k (x) \hat{v})$ is negative. For any $∥ \overset{\cdot}{x} ∥ > 0$ , there exist a matrix $0 < p ⩽ ∥ \overset{\cdot}{x} ∥$ such that $J_{x}^{T} \overset{\cdot}{x} ⩽ - p ∥ J_{x} ∥ < 0$ . Calculating the time derivative of $L (x, t)$ gives

\begin{matrix} \overset{\cdot}{L} (x, t) ⩽ - η^{2} λ_{\min} (q) ∥ x ∥^{2} + (η^{2} - 1) λ_{\min} (q) ∥ x ∥^{2} \\ + {\tilde{W}}_{c}^{T} [\frac{1}{2} K_{2} K_{2}^{T} - \frac{1}{4} (K_{1} K_{1}^{T} - K_{3} K_{3}^{T}) - {\bar{β}}_{1} β_{1}^{T} + {\bar{β}}_{2} σ_{c} (\hat{x} (t_{f}), 0)] {\tilde{W}}_{c} \\ + {\tilde{W}}_{c}^{T} (K_{2} K_{4}^{T} + {\bar{β}}_{1} ε_{HJI} + {\bar{β}}_{2} ε (t_{f})) - \frac{1}{4} W_{c}^{T} (K_{1} K_{1}^{T} - K_{3} K_{3}^{T}) W_{c} \\ + {\tilde{W}}_{c}^{T} [\frac{1}{2} (K_{1} K_{1}^{T} - K_{3} K_{3}^{T}) + {\bar{β}}_{2} {\tilde{σ}}_{c} (x (t_{f}), 0)] W_{c} - p ∥ J_{x} ∥ \\ + \frac{1}{2} K_{4} K_{4}^{T} + ∥ R_{1}^{- 1} ∥ (B^{2} g_{\max}^{2} + A^{2} \nabla_{x}^{2} σ_{c_{\max}}) ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} . \end{matrix}

Defining $ξ^{T} = [x^{T}, {\tilde{W}}_{c}^{T} {\bar{β}}_{1}]$ , the time derivative of $L (x, t)$ can be further calculated as

\begin{matrix} \overset{\cdot}{L} (x, t) ⩽ - ξ^{T} [\begin{matrix} η^{2} λ_{\min} (q) & 0 \\ Π_{22} \end{matrix}] ξ + [\begin{matrix} 0 & Π_{33} + Π_{44} \end{matrix}] ξ + λ_{1} \\ - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} - p ∥ J_{x} ∥ \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2}, \\ \overset{Δ}{=} - ξ^{T} E ξ + N ξ - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} + λ_{1} - p ∥ J_{x} ∥, \\ ⩽ - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} + \frac{∥ N ∥^{2}}{4 λ_{\min} (E)} + λ_{1} \\ - p ∥ J_{x} ∥ - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} + λ_{2} \\ ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2}, \end{matrix}

where

Π_{22} = - \frac{1}{2} K_{2} K_{2}^{T} + \frac{1}{4} K_{1} K_{1}^{T} - \frac{1}{4} K_{3} K_{3}^{T} - {\bar{β}}_{2} σ_{c} (\hat{x} (t_{f}), 0) + {\bar{β}}_{1} β_{1}^{T},

Π_{33} = K_{2} K_{4}^{T} + {\bar{β}}_{1} ε_{HJI} + {\bar{β}}_{2} ε (t_{f}),

Π_{44} = (\frac{1}{2} K_{1} K_{1}^{T} - \frac{1}{2} K_{3} K_{3}^{T} + {\bar{β}}_{2} {\tilde{σ}}_{c} (x (t_{f}), 0)) W_{c},

λ_{1} = \frac{1}{2} K_{4} K_{4}^{T} + W_{c}^{T} (- \frac{1}{4} K_{1} K_{1}^{T} + \frac{1}{4} K_{3} K_{3}^{T}) W_{c},

\begin{matrix} λ_{2} = B^{2} g_{\max}^{2} + A^{2} \nabla_{x}^{2} σ_{c_{\max}}, \\ {\bar{β}}_{1} = \frac{β_{1}}{{(1 - β_{1}^{T} β_{1})}^{2}}, {\bar{β}}_{2} = \frac{β_{2}}{{(1 - β_{2}^{T} β_{2})}^{2}} . \end{matrix}

For convenience, we denote

λ_{3} = \frac{∥ N ∥^{2}}{4 λ_{\min} (E)} .

Thus, $\overset{\cdot}{L} (x, t) < 0$ if the following conditions $(i)$ and $(ii)$ are satisfied

(I) $|| ξ || > \sqrt{\frac{λ_{1} + λ_{3}}{λ_{\min} (E)}} + \frac{|| N ||}{2 λ_{\min} (E)}$ ,

(II) $|| e_{r} (t) || ⩽ \sqrt{\frac{(1 - η^{2}) λ_{min} (q) {|| x ||}^{2} || R_{1} ||}{λ_{2} {|| {\hat{W}}_{c} ||}^{2}}}$ .

(2) $Σ (x, \hat{μ} ({\hat{x}}_{r}), \hat{v} (x)) = 1$ , according to the definition of $Σ$ , $J_{x}^{T} \overset{\cdot}{x} ⩾ 0$ . Hence, the time derivative of $L (x, t)$ becomes

\begin{matrix} \overset{\cdot}{L} (x, t) ⩽ - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} \\ + J_{x}^{T} (f (x) + g (x) \hat{μ} (x_{r}) + k (x) \hat{v} (x)) \\ - \frac{1}{2} {\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} ({\hat{x}}_{c}, t_{f} - t) g ({\hat{x}}_{r}) R_{1}^{- 1} g^{T} (x) J_{x} \\ - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} + λ_{1} \\ + \frac{1}{2} {\tilde{W}}_{c}^{T} \nabla_{x} σ_{c} (x, t_{f} - t) k (x) R_{2}^{- 1} k^{T} (x) J_{x} \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} + λ_{3}, \\ ⩽ J_{x}^{T} f (x) - \frac{1}{2} J_{x}^{T} g (x) R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{t} - t) W_{c} \\ + \frac{1}{2} J_{x}^{T} k (x) R_{2}^{- 1} k^{T} (x) \nabla_{x}^{T} σ_{c} (x, t_{f} - t) W_{c} \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} \\ - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} + λ_{3} \\ + λ_{1} - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} . \end{matrix}

From (20a) and (20b), it follows that

- \frac{1}{2} R_{1}^{- 1} g^{T} ({\hat{x}}_{r}) \nabla_{x}^{T} σ_{c} ({\hat{x}}_{r}, t_{f} - t) W_{c} = μ ({\hat{x}}_{r}) - ε_{u}^{*},

\frac{1}{2} R_{2}^{- 1} k^{T} (x) \nabla_{c}^{T} σ_{c} (x, t_{f} - t) W_{c} = v (x) - ε_{v}^{*} .

Then, the following inequalities are derived

\begin{matrix} \overset{\cdot}{L} (x, t) = - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} + λ_{3} + λ_{1} \\ - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2} + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} \\ + J_{x}^{T} (f (x) + g (x) μ ({\hat{x}}_{r}) + k (x) v (x)) - J_{x}^{T} (g (x) ε_{u}^{*} + k (x) ε_{v}^{*}) . \end{matrix}

In term of Assumption 2, there exist a constant $d_{M}$ such that $∥ g (x) ε_{u}^{*} + k (x) ε_{v}^{*} ∥ ⩽ d_{M}$ , and considering Lemma 2 yields

\begin{matrix} \overset{\cdot}{L} (x, t) ⩽ - J_{x}^{T} Λ (x) J_{x} + d_{M} ∥ J_{x} ∥ \\ - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} + λ_{3} \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} + λ_{1} - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2}, \\ ⩽ - λ_{\min} (Λ (x)) (∥ J_{x} ∥ - \frac{d_{M}}{2 λ_{\min} (Λ (x))})^{2} + \frac{d_{M}^{2}}{4 λ_{\min} (Λ (x))} \\ - λ_{\min} (E) {(∥ ξ ∥ - \frac{∥ N ∥}{2 λ_{\min} (E)})}^{2} + λ_{3} \\ + λ_{2} ∥ R_{1}^{- 1} ∥ ∥ {\hat{W}}_{c} ∥^{2} ∥ e_{r} (t) ∥^{2} + λ_{1} - (1 - η^{2}) λ_{\min} (q) ∥ x ∥^{2}, \\ < 0 . \end{matrix}

For convenience, we denote

λ_{4} = \frac{d_{M}^{2}}{4 λ_{\min} (Λ (x))} .

Hence, the time derivative of $L (x, t)$ is negative if the following conditions are satisfied

(i) $|| ξ || > \sqrt{\frac{λ_{1} + λ_{3}}{λ_{\min} (E)}} + \frac{|| N ||}{2 λ_{\min} (E)}$ ,

(ii) $|| e_{r} (t) || ⩽ \sqrt{\frac{(1 - η^{2}) λ_{\min} (q) {|| x ||}^{2} || R_{1} ||}{λ_{2} {|| {\hat{W}}_{c} ||}^{2}}}$ ,

(iii) $|| J_{x} || > \sqrt{\frac{λ_{4}}{λ_{\min} (Λ (x))}} + \frac{d_{M}}{2 λ_{\min} (Λ (x))} .$

Case $II$ : Events are triggered, that is, $\forall t_{r}, r = lh, l \in N$ . Based on (33), then the difference Lyapunov function is obtained

\begin{matrix} Δ L (x, t) = L ({\hat{x}}_{r + 1}) - L (x (t_{r + 1}^{-})), \\ = Δ L_{1} (t) + Δ L_{2} (t) + Δ L_{3} (t) + Δ L_{4} (t) . \end{matrix}

From Case $I$ , $\overset{\cdot}{L} (t) < 0$ for $\forall t \in [t_{r}, t_{r + 1})$ . Notice that the system states and cost function are continuous, then the following inequalities are found

\begin{matrix} Δ L_{1} (t) = V^{*} ({\hat{x}}_{r + 1}) - V (x (t_{r + 1}^{- 1})) ⩽ 0, \\ Δ L_{2} (t) = V^{*} ({\hat{x}}_{r + 1}) - V^{*} ({\hat{x}}_{r}), \\ Δ L_{3} (t) = J^{*} ({\hat{x}}_{r + 1}) - J^{*} (x (t_{r + 1}^{-})) ⩽ 0, \\ Δ L_{4} (t) = \frac{1}{2} ({\tilde{W}}_{c}^{T} ({\hat{x}}_{r + 1}) {\tilde{W}}_{c} ({\hat{x}}_{r + 1}) \\ - {\tilde{W}}_{c}^{T} (x (t_{r + 1}^{-})) {\tilde{W}}_{c} (x (t_{r + 1}^{-}))) ⩽ 0 . \end{matrix}

where $Δ L_{2} (t) = V^{*} ({\hat{x}}_{r + 1}) - V^{*} ({\hat{x}}_{r}) ⩽ - ν (∥ {\hat{e}}_{r + 1} (t_{r}) ∥)$ , $ν (\cdot)$ is a class-k function and ${\hat{e}}_{r + 1} (t_{r}) = {\hat{x}}_{r + 1} - {\hat{x}}_{r}$ . This means that Lyapunov function $L (x (t), t)$ is also decreasing at triggering instants $t = t_{r + 1}, r \in N .$

As a consequence, from (30), (31) and the event-triggered condition (32), we can derive that the nonlinear two-player ZS game system (1) and the estimation error of weight vector ${\tilde{W}}_{c}$ are UUB.

Application in the missile-target interception system

To demonstrate the efficiency of the presented algorithm in Section 4, the missile-target interception system is introduced. The motion differential equations of the missile and the target are, respectively, given by

{\begin{matrix} {\overset{\cdot}{x}}_{M} = V_{M} \cos α, \\ {\overset{\cdot}{y}}_{M} = V_{M} \sin α, \\ \overset{\cdot}{α} = a_{M} / V_{M}, \\ {\overset{\cdot}{a}}_{M} = (u - a_{M} /) τ_{M}, \end{matrix}

(35)

where $(x_{M}, y_{M})$ is the position coordinates of the missile in inertial ordinate system, $a_{M}$ presents the missile lateral acceleration, and $τ_{M}$ denotes the missile autopilots time constant.

{\begin{matrix} {\overset{\cdot}{x}}_{T} = V_{T} \cos β, \\ {\overset{\cdot}{y}}_{T} = V_{T} \sin β, \\ \overset{\cdot}{β} = a_{T} / V_{T}, \\ {\overset{\cdot}{a}}_{T} = (v - a_{T} /) τ_{T}, \end{matrix}

(36)

where $(x_{T}, y_{T})$ is the position coordinates of the target in inertial ordinate system, $a_{T}$ presents the target lateral acceleration, and $τ_{T}$ denotes the target autopilots time constant.

This section consider the geometry of planar interception as provided in Figure 2, which is borrowed from (Sun and Liu, 2018c), where $(X, Y)$ indicates position coordinates. Accordingly, the relative kinematics equation of the missile-target intercept system is obtained as follows, and some related notations will be used to simplify the upcoming description (see Table 2)

{\begin{matrix} \overset{\cdot}{r} = V_{T} \cos (β - θ) - V_{M} \cos (α - θ), \\ r \overset{\cdot}{θ} = V_{T} \sin (β - θ) - V_{M} \sin (α - θ), \\ \overset{\cdot}{α} = u / V_{M}, \\ \overset{\cdot}{β} = v / V_{T}, \end{matrix}

(37)

Figure 2.

Engagement geometry.

Table 2.

Some related notations.

Notations	Meaning
$V_{M}$	the missile velocity
$V_{M}$	the target velocity
$u$	the missile control vector perpendicular to the velocity vector
$v$	the target control vector perpendicular to the velocity vector
$α$	is the missile flight-path angle (FPA)
$β$	the target FPA
$θ$	is the line-of-sight (LOS) angle
$\overset{\cdot}{θ}$	is the LOS angle rate
$r$	the relative distance between the missile and the target
$\overset{\cdot}{r}$	the range rate along the LOS

Taking the time derivative of the LOS angle rate $\overset{\cdot}{θ}$ , and bearing (37) in mind, we get

\overset{\cdot\cdot}{θ} = - \frac{2 \overset{\cdot}{r}}{r} \overset{\cdot}{θ} - \frac{\cos (α - θ)}{r} u + \frac{\cos (β - θ)}{r} v .

(38)

To accomplish the goal of intercepting maneuverable aircraft, the relative distance $r$ will be converged to zero by adjusting the missile control vector $u$ . Obviously, in differential equation (38), the drift dynamics will go to infinity when $r$ converges to zero. Further, equation (38) is the problem of finite-time missile-target interception by nature, and does not satisfy with the condition of local Lipschitz.

Therefore, the above dynamic equations need to be performed the following mathematical transformations, and the necessary definition is introduced.

Definition 1: (Bardhan and Ghose, 2015) Assume that the closest distance between the missile and the target at time t, if both the missile and the target are uncontrolled from that instant onwards, then the zero effort miss (ZEM) can be express by $r_{miss} (t)$

r_{miss} (t) = \frac{r^{2} \overset{\cdot}{θ}}{\sqrt{{\overset{\cdot}{r}}^{2} + r^{2} {\overset{\cdot}{θ}}^{2}}} .

(39)

Remark 6: As shown in (39), the missile will successfully capture the target in the terminal guidance phase, if the following conditions are satisfied

$(i)$ : the LOS angle rate $\overset{\cdot}{θ}$ goes to zero.

$(ii)$ : the range rate along the LOS is negative.

Define the time-to-go, a new state variable and a new time variable are, respectively, $t_{go} = - r / \overset{\cdot}{r}$ , $\tilde{x} = [{\tilde{x}}_{1}, {\tilde{x}}_{2}]^{T} = [θ, \overset{\cdot}{θ} t_{go}]^{T}$ , and $\tilde{t} = \ln (r (0)) - \ln (r (t))$ . Taking the derivative of $\tilde{x}$ with respect to $\tilde{t}$ , the following new dynamics equation is obtained

\overset{\cdot}{\tilde{x}} (\tilde{t}) = f (\tilde{x}) + g (\tilde{x}) \tilde{u} + k (\tilde{x}) \tilde{v},

(40)

where

\begin{array}{l} \begin{array}{l} f (\tilde{x}) = [\begin{matrix} {\tilde{x}}_{2} \\ {\tilde{x}}_{2} + {\tilde{x}}_{2}^{3} \end{matrix}], \tilde{u} = t_{g o} u, \tilde{v} = t_{g o} v, \end{array} \\ \begin{array}{l} g (\tilde{x}) = [\begin{matrix} 0 \\ \frac{1}{\overset{\cdot}{r}} (\cos (α - θ) + {\tilde{x}}_{2} \sin (α - θ)) \end{matrix}], \end{array} \\ k (\tilde{x}) = [\begin{matrix} 0 \\ \frac{- 1}{\overset{\cdot}{r}} (\cos (β - θ) + {\tilde{x}}_{2} \sin (β - θ)) \end{matrix}] . \end{array}

It is clear that dynamics equation (46) is controlled for all $(α - θ) \neq π / 2$ , $(β - θ) \neq π / 2$ . Therefore, the guidance law applies to the following domain

Φ = {x : | (α - θ) | \neq π / 2, | (β - θ) | \neq π / 2, r \neq 0, V_{r} < 0} .

Remark 7: The problem of finite time missile-target interception in (38) is converted into an infinite time version in (40) by mathematical manipulation. In (40), $\tilde{t}$ tend to be infinity with $r$ decrease to zero that reveal that the guidance problem is converted into a problem of two-player ZS differential games.

Remark 8: It is clear that the nonlinear dynamics $k (\tilde{x})$ and $g (\tilde{x})$ are bounded in kinematics equation (40), which show that the Assumption 2 is easy to satisfy in actual engineering.

Missile-target interception simulation

For simplicity, let the velocity of the missile and the target be constants during the terminal guidance. Assume that the missile and the target velocity are $V_{M} = 750 m / s$ and $V_{T} = 450 m / s$ , respectively. Given the time constants of the missile and the target as $τ_{M} = 0.1 s$ and $τ_{T} = 0.1 s$ . Let the initial FPA of the missile and the target are $α_{0} = 65 °$ and $β_{0} = 80 °$ . Assume the initial position parameters are $(x_{M}, y_{M}) = (0, 0)$ and $(x_{T}, y_{T}) = (3500, 3400)$ , and the initial LOS angler is $θ_{0} = 0 °$ . To realize the designed algorithm of the event-triggered differential game, we choose the time-dependent term $Q (\tilde{x}, t) = 20 ({\tilde{x}}_{1}^{2} + {\tilde{x}}_{2}^{2} + τ^{2})$ , where $τ = t_{f} - t$ and $t_{f} = 40 s$ . Let the $R_{1} = 40$ , $R_{2} = 0.01$ , and $Ψ (\tilde{x} (t_{f}), t_{f}) = {\tilde{x}}^{T} (t_{f}) \tilde{x} (t_{f})$ denotes the terminal constraint.

For the critic network, a time-dependent activation function with $L = 8$ neurons is selected as $σ_{c} (\tilde{x}, t_{f} - t) = [{\tilde{x}}_{1}^{2} e^{- τ}, {\tilde{x}}_{2}^{2} e^{- τ}, {\tilde{x}}_{1} {\tilde{x}}_{2} τ, {\tilde{x}}_{1}^{4} e^{- τ}, {\tilde{x}}^{2} e^{- τ}, {\tilde{x}}_{1}^{3} {\tilde{x}}_{2}, {\tilde{x}}_{1}^{2} {\tilde{x}}_{2}, {\tilde{x}}_{1} {\tilde{x}}_{2}^{2}]^{T}$ , and the critic network weight is chosen as ${\hat{W}}_{c} = [{\hat{W}}_{c 1}, {\hat{W}}_{c 2}, {\hat{W}}_{c 3}, {\hat{W}}_{c 4}, {\hat{W}}_{c 5}, {\hat{W}}_{c 6}, {\hat{W}}_{c 7}, {\hat{W}}_{c 8}]^{T}$ in which the initial value of ${\hat{W}}_{c}$ is $[50, 50, 50, 50, 50, 50, 50, 50]^{T}$ . The learning rate and the Lyapunov function defined in Lemma 2 are set as $α_{c} = 0.02$ and $J (\tilde{x}) = 0.5 ({\tilde{x}}_{1}^{2} + {\tilde{x}}_{2}^{2})$ . The sampling period of the time-triggering scheme is given as $h = 0.02 s .$ Finally, a small pumping signal $n (t) = \overset{5}{\sin} (t) \cos (t) + \overset{5}{\sin} (2 t) \cos (0.2 t)$ is added to the control input in the first 10 seconds to ensure the persistent excitation effect.

The PETADP algorithm presented in this paper is used to the missile-target interception system, then the following Figures 3 –11 is obtained. As show in Figures 3–4, the LOS angle rate $\overset{\cdot}{θ}$ is converge to the neighbourhood of zero, and the range rate along the LOS is always negative, that is, $\overset{\cdot}{r} < 0$ . Obviously, the maneuvering target is successfully captured by the missile in Figure 2 since both $\overset{\cdot}{θ}$ and $\overset{\cdot}{r}$ satisfy the capture criteria in Remark 5. Figure 5 and Figure 6 reflect the lateral accelerations $a_{T}$ and $a_{M}$ . Figure 7(a) and (b) show the relative distance $r$ between the missile and the target under the time-triggered scheme and the PET scheme, respectively. It is clear that relative distance $r < 3$ in Figure 7(b) while $r < 10$ in Figure 7(a), which illustrated that the the periodic event-triggered ADP technique can implement more precise control for missiles. Figure 8 reflects the trajectory of engagement between the missile and the target, and the trajectories of ${\hat{W}}_{c}$ is given in Figure 9. Figure 10 illustrates that the minimum triggering intervals $τ_{s} = 0.02$ under the PET scheme, therefore the Zeno behavior is eliminated. Obviously, this scheme combines the advantages of both traditional event-triggered schemes and the periodic sampling, the benefits of saved communication resources and computing costs are preserved while the Zeno behavior is avoid. In Figure 11, the periodic event-triggered ADP technique needs only 492 samples, while the traditional period sampling ADP technique and continuous event-triggered ADP technique require 750 samples and 551 samples, respectively. By calculating, the number of controller updates have been saved by $34.4 %$ under the the periodic event-triggered ADP technique. Thus, we conclude that the PET controller is highly efficient in decreasing the frequency of data transmission and saving compute costs.

Figure 3.

The LOS angular rate.

Figure 4.

The range rate along the LOS.

Figure 5.

The lateral acceleration of target.

Figure 6.

The lateral acceleration of missile.

Figure 7.

The relative distance under (a) the time-triggered scheme; (b) the periodic event-triggered scheme.

Figure 8.

Engagement trajectories.

Figure 9.

The critic weights.

Figure 10.

Triggering intervals under (a) the continuous event-triggered scheme; (b) the periodic event-triggered scheme.

Figure 11.

Computation results of sampling numbers.

Remark 9: The above simulation results rendering that the signals of missile-target interception system are divergent at the terminal guidance time. In fact, the simulation curves are inevitable divergent at terminal time of the missile interception, and the major reason is that the system parameter value changes abruptly at the guidance terminal. In particular, the guidance law designed in this article is no longer applicable when the relative distance between the missile and the target tends to zero.

Remark 10: As seen in Figures 10–11, the PET scheme proposed in this paper can further save communication resources as well as the Zeno behavior is avoid compared to the continuous ET scheme used in Sahoo et al. (2017), Vamvoudakis (2014), Zhong et al. (2015), Zhu et al. (2017) and Wang et al. (2016), in which the minimum triggered interval $τ_{s} = 0.02$ .

Conclusions

The issues of finite-time optimal control for event-triggered two-player ZS differential games have been investigated in this article. A periodic event-triggered ADP algorithm has been established to save communication costs while eliminate the undesired Zeno behavior, for which the control inputs are updated only when the designed ET condition was satisfied. To implement this algorithm, a single CNN and a time-varying activation function have been proposed to approximate the Nash equilibrium solution of the time-dependent HJI equation. Further, the terminal estimation error has been minimized to deal with the finite-horizon constraints, thereby a ET critic weight update law has been proposed. By virtue of the Lyapunov function method, some stability criteria have been obtained to guarantee the UUB for the event-triggered closed-feedback system and the weight error of the single CNN, respectively. Eventually, a missile-target interception system with finite time constraints is introduced to demonstrate the efficiency of the proposed methods. Recently, robust control of differential games receives extensive attention (Sun et al., 2016). In fact, many model errors and external disturbances widely exist in the modern missile-target interception system. Therefore, future efforts will focus on designing robust optimal controller for finite-horizon differential games.

Footnotes

Declaration of conflicting interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by National Natural Science Foundation of China under Grant 61473147.

ORCID iDs

Dandan Duan

Chunsheng Liu

References

Bardhan

Ghose

(2015) Nonlinear diifferential games-based impact-angle-constrained guidance law. Journal of Guidance Control and Dynamics 38(3): 1–19.

Bhasin

Kamalapurkar

Johnson

, et al. (2013) A novel actor-critic-identifier architecture for approximate optimal control of uncertain nonlinear systems. Automatica 49(1): 82–92.

Cavalieri

Satak

Hurtado

(2013) Incomplete information pursuit-evasion games with uncertain relative dynamics. In: AIAA Guidance, Navigation, and Control Conference, National Harbor, MD, USA, 13–17 January 2014, pp. 1–8. Reston, VA, USA: American Institute of Aeronautics and Astronautics.

Cui

Zhang

Luo

(2016) Online finite-horizon optimal learning algorithm for nonzero-sum games with partially unknown dynamics and constrained inputs. Neurocomputing 185: 37–44.

Dai

Liu

Sun

(2019) Adaptive optimal fault-tolerant control scheme for a class of strict-feedback nonlinear systems. Transactions of the Institute of Measurement and Control 41(4): 1079–1087.

Dierks

(2012) Online optimal control of affine nonlinear discrete-time systems with unknown internal dynamics by using time-based policy update. IEEE Transactions on Neural Networks and Learning Systems 23(7): 1118–1129.

Ding

Liu

(2014) Neural-network-based robust optimal control design for a class of uncertain nonlinear systems via adaptive dynamic programming. Information Sciences an International Journal 282: 167–179.

Chai

(2017) Online solution of two-player zero-sum games for continuous-time nonlinear systems with completely unknown dynamics. IEEE Transactions on Neural Networks and Learning System 27(12): 2577–2587.

Gao

Liu

(2018) Adaptive neural network-based control for a class of nonlinear pure-feedback systems with time-varying full state constraints. IEEE/CAA Journal of Automatica Sinica 5(5): 923–933.

10.

Guo

Zhou

(2019) An observer-based continuous adaptive sliding mode guidance against chattering for homing missiles. Transactions of the Institute of Measurement and Control 41(12): 3309–3320.

11.

Lee

(2018) Optimal proportional-integral guidance with reduced sensitivity to target maneuvers. IEEE Transactions on Aerospace and Electronic Systems 54(5): 2568–2579.

12.

Heydari

Balakrishnan

(2013) Finite-horizon control-constrained nonlinear optimal control using single network adaptive critics. IEEE Transactions on Neural Networks and Learning System 24(1): 145–157.

13.

Huang

(2016) Neuro-observer based online finite-horizon optimal control for uncertain non-linear continuous-time systems. IET Control Theory and Applications 11(3): 401–410.

14.

Jagat

Sinclair

(2017) Nonlinear control for spacecraft pursuit-evasion game using the state-dependent riccati equation method. IEEE Transactions on Aerospace Electronic Systems 53(6): 3032–3042.

15.

Kishi

Bettwy

(1965) Optimal and suboptimal design of proportional navigation systems. In: Recent Advances in Optimization Techniques. New York: John Wiley.

16.

(2016) A three-dimensional robust nonlinear terminal guidance law with ISS finite-time convergence. Internationa Journal of Control 89(5): 938–949.

17.

Liu

Tong

(2018a) Fuzzy based multi-error constraint control for switched nonlinear systems and its applications. IEEE Transactions on Fuzzy Systems 27(8): 1519–1531.

18.

Liu

Gao

Tong

(2015) Fuzzy approximation-based adaptive backstepping optimal control for a class of nonlinear discrete-time systems with dead-zones. IEEE Transactions on Fuzzy Systems 24(1): 16–28.

19.

Liu

Tong

, et al. (2018b) Adaptive control-based barrier lyapunov functions for a class of stochastic nonlinear systems with full state constraints. Automatica 87: 83–93.

20.

Liu

Tong

Wang

(2009) Observer-based direct adaptive fuzzy control of uncertain nonlinear systems and its applications. International Journal of Control, Automation and Systems 7(4): 681–690.

21.

Liu

Zeng

Tong

, et al. (2019) Adaptive neural network control for active suspension systems with time-varying vertical displacement and speed constraints. IEEE Transactions on Industrial Electronics 66(12): 9458–9466.

22.

Wang

(2018) Approximate-optimal control algorithm for constrained zero-sum differential games through event-triggering mechanism. Nonlinear Dynamics 95(4): 2639–2657.

23.

Parras

Del Val

Zazo

(2016) A new approach for solving anti-jamming games in stochastic scenarios as pursuit-evasion games. In: 2016 IEEE Statistical Signal Processing Workshop (SSP), Palma, Spain, 26–29 Jun 2016, pp. 1–5. NEW YORK, NY, USA: IEEE.

24.

Perelman

Shima

Rusnak

(2011) Cooperative differential games strategies for active aircraft protection from a homing missile. Journal of Guidance Control and Dynamics 34(3): 761–773.

25.

Zhang

Rui

Yang

(2018) Neural network-based

H_{\infty}

sliding mode control for nonlinear systems with actuator faults and unmatched disturbances. Neurocomputing 275: 2009–2018.

26.

Rufus

(1954) Differential games III: the basic principles of the solution process. Rand Corp Santa Monica Ca, Technical Report RM-1411-PR, Rand Corporation, 1954.

27.

Sahoo

S Jagannathan

(2015) Neural network-based event-triggered state feedback control of nonlinear continuous-time systems. IEEE Transactions on Neural Networks and Learning Systems 27(3): 497–509.

28.

Sahoo

Jagannathan

(2017) Approximate optimal control of affine nonlinear continuous-time systems using event-sampled neurodynamic programming. IEEE Transactions on Neural Networks and Learning Systems 28(3): 639–652.

29.

Shima

Golan

(2007) Linear quadratic differential games guidance law for dual controlled missiles. IEEE Transactions on Aerospace and Electronic Systems 43(3): 834–842.

30.

Song

(2019) Robust impact angle constraints guidance law with autopilot lag and acceleration saturation consideration. Transactions of the Institute of Measurement and Control 41(1): 182–192.

31.

Sun

Liu

(2018a) Finite-horizon differential games for missile-target interception system using adaptive dynamic programming with input constraints. International Journal of Systems Science 49(2): 1–20.

32.

Sun

Liu

(2018b) Decentralised zero-sum differential game for a class of large-scale interconnected systems via adaptive dynamic programming. International Journal of Control 92(12): 2917–2927.

33.

Sun

Liu

(2018c) Finite-horizon diifferential games for missile-target interception system using adaptive dynamic programming with input constraints. International Journal of Systems Science 49(2): 264–283.

34.

Sun

Liu

(2016) Robust differential game guidance laws design for uncertain intercepter-target engagement via adaptive dynamic programming. Internationa Journal of Control 90(5): 990–1004.

35.

Tang

Wen

Liu

(2014) Power system stability control for a wind farm based on adaptive dynamic programming. IEEE Transactions on Smart Grid 6(1): 166–177.

36.

Vamvoudakis

(2014) Event-triggered optimal adaptive control algorithm for continuous-time nonlinear systems. IEEE/CAA Journal of Automatica Sinica 1(3): 282–293.

37.

Vamvoudakis

Lewis

(2011) Multi-player non-zero-sum games: Online adaptive learning solution of coupled hamiltoncjacobi equations. Automatica 47(8): 1556–1569.

38.

Wang

Liu

Wei

(2011) Adaptive dynamic programming for finite-horizon optimal tracking control of a class of nonlinear systems. In: Proceedings of the 30th Chinese Control Conference, Yantai, China, 22–24 July 2011, pp. 2450–2455. Piscataway, NJ, USA: IEEE.

39.

Wang

Liu

Wei

(2012) Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach. Neurocomputing 78(1): 14–22.

40.

Wang

Liu

(2017) Event-driven adaptive robust control of nonlinear systems with uncertainties through ndp strategy. IEEE Transactions on Systems Man and Cybernetics Systems 47(7): 1358–1370.

41.

Wang

Zhang

Liu

(2016) Event-based input-constrained nonlinear

h_{\infty}

state feedback with adaptive critic and neural implementation. Neurocomputing 241: 848–856.

42.

Wang

(2017) Adaptive smooth second-order sliding mode control method with application to missile guidance. Transactions of the Institute of Measurement and Control 39(6): 848–860.

43.

Wei

Liu

Lewis

, et al. (2017) Mixed iterative adaptive dynamic programming for optimal battery energy control in smart residential microgrids. IEEE Transactions on Industrial Electronics 64(5): 4110–4120.

44.

White

Sofge

Reinhold

(1994) Handbook of intelligent control: neural, fuzzy, and adaptive approaches. IEEE Transactions on Neural Networks 7(5): 851–852.

45.

Yang

Liu

Huang

(2013) Neural-network-based online optimal control for uncertain nonlinear continuoustime systems with control constraints. IET Control Theory and Applications 7(17): 2037–2047.

46.

Zhang

Zhao

Zhu

(2017a) Data-driven adaptive dynamic programming for continuous-time fully cooperative games with partially constrained inputs. Neurocomputing 238: 377–386.

47.

Zhang

Zhao

Zhu

(2017b) Event-triggered

h_{\infty}

control for continuous-time nonlinear system via concurrent learning. IEEE Transactions on Systems Man and Cybernetics Systems 47(7): 1071–1081.

48.

Zhao

Jagannathan

(2015) Neural network-based finite-horizon optimal control of uncertain affine nonlinear discrete-time systems. IEEE Transactions Neural Netw Learn System 26(3): 486–499.

49.

Zhong

(2015) Event-triggered adaptive dynamic programming for continuous-time nonlinear system using measured input-output data. In: 2015 International Joint Conference on Neural Networks (IJCNN), Killarney, Ireland, 12–17 July 2015, pp. 1–8. NEW YORK, NY, USA: IEEE.

50.

Zhu

Zhao

(2017) Event-triggered optimal control for partially-unknown constrained-input systems via adaptive dynamic programming. IEEE Transactions on Industrial Electronics 64(5): 4101–4109.