Predefined-time formation tracking for AAVs with prescribed performance: A neural network-based reinforcement learning approach

Abstract

This paper proposes a solution to the predefined-time formation tracking problem of autonomous aerial vehicles (AAVs). To enhance the systems’ coordination performance and convergence rate, a predefined-time hierarchical control (PTHC) framework is developed. Furthermore, a prescribed-performance control mechanism is introduced to impose prior constraints on both transient and steady-state behaviors during the design stage, thereby ensuring that the tracking error remains within predefined bounds. For the state estimation process, a predefined-time distributed estimator based on nonsingular sliding mode control is designed, effectively avoiding the singularity issues commonly encountered in conventional sliding mode approaches. In the control implementation, an adaptive strategy integrating neural networks with reinforcement learning is proposed to improve the systems’ resistance to external disturbances, actuator faults, and modeling uncertainties. This combination significantly enhances the robustness and reliability of the controller, ensuring stable operation under complex and dynamic environments. The designed control strategy ensures the realization of the desired formation tracking goal within the predefined time. The paper further establishes a set of sufficient criteria to ensure predefined-time stability. Finally, extensive simulation studies are conducted to verify the proposed control methods’ efficiency, robustness, and practical applicability.

Keywords

autonomous aerial vehicles (AAVs)nonsingular sliding mode control reinforcement learning (RL)actor-critic neural network prescribed performance

Introduction

A multi-agent system (MAS) comprises numerous intelligent agents that exchange information and collaborate to complete complex tasks through distributed sensing, cooperative decision-making, and coordinated control (Gao et al., 2024a; Gu et al., 2025; Han et al., 2017b; Liu et al., 2025). As one of the most important types of intelligent agents, an autonomous aerial vehicle (AAV) is capable of flying autonomously or according to remote control commands (Han et al., 2016, 2017a). However, a single AAV has certain limitations in terms of payload capacity, mission range, and fault tolerance, making it difficult to independently complete complex engineering tasks. To overcome these limitations, cooperative operation among multiple AAVs has emerged, leveraging inter-agent communication and task sharing to achieve more-efficient group coordination. Multi-AAV systems can flexibly adapt to changing environments or mission requirements, and each AAV can dynamically adjust its role, trajectory, or behavior according to real-time conditions. This flexibility significantly enhances the applicability of AAV systems in a wide range of scenarios. In recent years, cooperative formation control technology for multiple AAVs, as a key enabling technique for multi-AAV cooperative missions, has become a prominent research topic and has attracted increasing attention (Jin et al., 2024; Zhang and Duan, 2024). Nevertheless, existing methods still suffer from limited adaptability and robustness when operating in complex environments, which provides important motivation for further research.

Cooperative time-varying formation tracking (TVFT) control for AAVs has recently become a key research focus (Wei et al., 2023; Wu et al., 2023b; Yan et al., 2023), with the primary goal of enhancing overall system performance. Among the existing approaches, Yan et al. (2023) propose an event-triggered formation control method for discrete-time MAS, aiming to achieve efficient and stable formation flying of multiple UAVs under time-delay conditions. Wei et al. (2023) propose a robust UAV formation control strategy that integrates a cost-based switching law, dynamic event-triggered mechanism, and disturbance observers to maintain stable consensus under failures and communication constraints. However, in practical and complex environments, such methods may still face limitations in adaptability. Consequently, time-varying formation control strategies with higher robustness and flexibility have attracted increasing attention.

Achieving a rapid convergence rate is crucial in real-time formation control systems. Conventional control methods that rely on asymptotic stability can merely ensure exponential convergence, which fails to satisfy the stringent requirements of AAVs for fast response and high-precision performance. To address this issue, the finite-time stability theory has been widely adopted (Ding et al., 2024; Wu et al., 2023a), enabling system convergence within a finite time. However, the main drawback of such methods lies in their dependence on initial conditions, that is, when the initial states are large, the convergence time may increase significantly. To overcome this limitation, the fixed-time control method (Meng et al., 2023; Miao et al., 2024) was developed, ensuring that the system converges within a constant time bound that does not rely on the initial conditions. Nevertheless, this time bound is implicitly determined by the system parameters and cannot be specified in advance during the design stage. However, the predefined-time control approach extends the fixed-time method by allowing the designer to specify the convergence time explicitly during the design stage. This feature enhances system predictability, task-scheduling efficiency, and adaptability to diverse mission requirements. At present, several predefined-time consensus and formation control methods for AAVs have been proposed (Li et al., 2025; Yang et al., 2024). However, when introducing predefined performance constraints and reinforcement learning (RL) mechanisms, it remains challenging to simultaneously ensure rapid convergence, stability, and robustness. Therefore, investigating RL-based predefined-performance formation control methods is of great significance for enhancing the intelligence and real-time control capability of AAVs.

In addition, the concept of prescribed performance control (PPC) has been increasingly adopted in recent years to explicitly constrain the transient and steady-state behaviors of system-tracking errors within predefined time-varying boundaries (Yang et al., 2024; Zhang et al., 2024; Zhou et al., 2024). By introducing performance functions, PPC provides a systematic way to shape error evolution, ensuring that tracking errors remain within user-specified bounds at all times. When integrated with predefined-time control, this combination enables not only bounded tracking error evolution but also guaranteed convergence within a user-specified time horizon. However, the integration of PPC into predefined-time formation control for AAVs has not been fully explored.

Although such methods improve the system’s robustness to some extent, existing studies still fail to fully address the key control challenges in AAVs. For MASs, various control schemes based on predefined-time sliding mode surfaces have been proposed to ensure that system states converge within a predefined time (Guan et al., 2025; Jia et al., 2023; Song et al., 2024). However, these approaches generally suffer from singularity issues. Specifically, when the system state approaches the origin of the sliding surface, the control input may become unbounded, thereby hindering practical implementation and even leading to system instability. This problem becomes more severe in systems characterized by complex nonlinear dynamics and strong coupling. In particular, for AAVs, nonlinear behaviors, external disturbances, and communication uncertainties make the direct application of existing control methods impractical. To overcome these challenges, this paper introduces a predefined-time nonsingular sliding mode control mechanism into the observer design. This mechanism not only enables fast estimation of unknown states but also effectively avoids divergence in control inputs caused by singularities. Consequently, both estimation accuracy and system robustness are significantly improved, providing a more practical control strategy for AAVs operating in complex environments.

Recently, neural networks (NNs) have shown strong nonlinear approximation capabilities and have been widely applied in control design for systems with unknown dynamics (Gao et al., 2024b; Greene et al., 2023; Li et al., 2023). Compared to traditional methods, NNs provide better adaptability and generalization capabilities, making them suitable for complex systems subject to environmental disturbances. Motivated by these developments, RL has attracted increasing attention as a label-free control methodology that learns decision-making policies from interactions with the environment. The actor-critic framework, consisting of an actor network responsible for producing control actions and a critic network for assessing and improving the policy, has been shown to effectively enhance the robustness and control accuracy of nonlinear systems. Studies have successfully applied RL to complex dynamic systems (Gong and Yang, 2024; Ma et al., 2023; Wang et al., 2023). However, most RL methods are designed based on finite-time or asymptotic stability frameworks, with convergence times depending on initial conditions, and they lack integration with predefined-time control, making them less suitable for tasks with strict time constraints. Therefore, combining RL with predefined-time control to achieve user-defined convergence times remains an open challenge.

Motivated by the above analysis, this paper investigates the predefined-time TVFT problem for multiple AAVs. Serving as a substantial theoretical extension of our preliminary investigations into networked marine vehicles (Zhao et al., 2025), this manuscript addresses a significantly more demanding control scenario. Unlike the study by Zhao et al. (2025), which mainly focused on the basic convergence of the system without strictly imposing prior constraints on transient tracking behaviors and relied on estimators that may suffer from singularity issues, this paper addresses the more challenging issue of achieving strict transient and steady-state performance guarantees. To this end, considering the strong nonlinearity and coupling characteristics of multi-AAV systems, a novel predefined-time hierarchical control (PTHC) framework is employed to decompose the complex TVFT problem into two tractable subproblems. Specifically, the proposed strategy integrates prescribed-performance constraints, an NN-based RL mechanism, and a nonsingular sliding mode estimator. The main contributions of this paper are summarized as follows:

To address the control challenges of multiple AAVs, this paper proposes a hierarchical predefined-time TVFT control algorithm. The proposed control strategy is designed to effectively handle model uncertainties, time-varying disturbances, and malicious attacks, enabling the AAV swarm to be flexibly configured according to mission requirements in aerial surveillance and cooperative combat tasks, thereby achieving fast convergence and stable coordination performance.

Unlike the method in Zhao et al. (2025), this paper introduces prescribed performance constraints, which effectively ensure both the transient and steady-state performance of the system and enable the controller to achieve adaptive regulation within the system. On this basis, an NN is further integrated with the actor-critic RL framework to construct a computationally efficient control scheme with online learning capability. Benefiting from the introduction of prescribed performance constraints, the proposed scheme achieves fast response, real-time control, and high-precision tracking in complex environments, demonstrating significant engineering application value.

Unlike the methods in Gong and Yang (2024) and Zhao and Yang (2024), this paper innovatively integrates RL with observer-based predefined-time sliding mode control. This approach enables the explicit setting of the convergence time during the design phase and ensures that the system converges within the specified time through controller parameter tuning, while simultaneously enhancing steady-state performance.

Notations

Let $\nabla$ be the gradient operator. The N-order identity matrix is denoted by $I_{N}$ , while $1_{N}$ represents an N-dimensional column vector consisting of ones. The sets of real and positive real numbers are signified by ℝ and $ℝ_{+}$ , respectively, with $ℝ^{N}$ representing the N-dimensional Euclidean space. The symbol ⊗ designates the Kronecker products, and $diag {\cdot}$ is utilized for diagonal matrix representation.

Preliminaries

Graph theory

The communication topology of the N-vehicle system is modeled by a digraph $E = (V, E, A)$ . In this topology, $V = {1, 2, \dots, N}$ defines the vertices, $E \subseteq V \times V$ stands for the information links, and $A = {[a_{ij}]}_{N \times N}$ represents the weighted adjacency matrix. The neighbor set of node i is defined as $N_{i} = {j \in V ∣ (i, j) \in E}$ . The graph’s Laplacian matrix $L = {[l_{ij}]}_{N \times N}$ is constructed by $L = D - A$ , where $D = diag (d_{1}, \dots, d_{N})$ with $d_{i} = \sum_{j = 1}^{N} a_{ij}$ being the in-degree matrix. The specific elements of L follow $l_{ii} = \sum_{j \in N_{i}} a_{ij}$ and $l_{ij} = - a_{ij}$ ( $i \neq j$ ).

For generality, the set of vehicles is partitioned into two disjoint subgroups, denoted by $E_{l} = (V_{l}, E_{l}, A_{l}), l \in {1, 2},$ where $V_{r} \cap V_{m} = \emptyset$ for $r \neq m \in {1, 2}$ and $⋃_{l = 1}^{2} V_{l} = V$ .

A diagonal matrix $B = diag (b_{1}, b_{2}, \dots, b_{N})$ is defined. Here, $b_{i} > 0$ if vehicle i has direct access to its leader, and $b_{i} = 0$ otherwise. In addition, the Laplacian matrix L is reformulated to describe the communication topology of the networked system as:

L = [\begin{matrix} L_{11} & L_{12} \\ L_{21} & L_{22} \end{matrix}],

A can be expressed as:

A = [\begin{matrix} A_{11} & A_{12} \\ A_{21} & A_{22} \end{matrix}],

where $A_{rr}$ characterizes the internal communication among the vehicles within group $V_{r}$ , while $A_{rm}$ describes information flow transmitted from the vehicles in $V_{m}$ to those in $V_{r}$ ( $r \neq m$ , $r, m \in {1, 2}$ ).

Assumption 1. (Shao and Zheng, 2022) The directed graph Ê, partitioned acyclically, is assumed to be in-degree balanced, that is, $1^{T} L_{kj} = 0$ holds for all $k \neq j \in {1, 2}$ .

Lemma 1. (Ren et al., 2021) Suppose that $V (x)$ is a continuously differentiable function satisfying the following condition:

\overset{\cdot}{V} (x) = - aV (x) - b ψ_{ℓ} (t) V (x), t \in [t_{0}, T_{ℓ}],

for some positive constants a and b. In this context, the time-varying parameter $ψ_{ℓ} (t)$ is formulated as:

ψ_{ℓ} (t) = {\begin{matrix} \frac{{\overset{\cdot}{κ}}_{ℓ} (t)}{κ_{ℓ} (t)}, & t_{0} \leq t < T_{ℓ}, \\ \frac{ρ}{T_{ℓ}}, & t \geq T_{ℓ}, \end{matrix}

alongside the auxiliary function $κ_{ℓ} (t)$ given by $κ_{ℓ} (t) = T_{ℓ}^{ρ} ∕ {(T_{ℓ} - t)}^{ρ}$ for $t \in [t_{0}, T_{ℓ})$ , subject to the parameters $ρ > 1$ , $t_{0} > 0$ , and $T_{ℓ} > 0$ . $V (x)$ converges within $T_{ℓ}$ , and

{\begin{matrix} V (x) \leq κ_{ℓ} {(t)}^{- ϱ} \exp (- ν (t - t_{0})) V (x_{0}), & t \in [t_{0}, T_{ℓ}), \\ V (x) \equiv 0, & t \in [T_{ℓ}, \infty) . \end{matrix}

Model formulation

In the coordinated control of multiple AAVs, a commonly adopted approach is the inner–outer loop hierarchical structure. Specifically, the outer loop is responsible for generating the desired trajectory and attitude, while the inner loop achieves attitude stabilization and tracking through appropriate attitude controllers. For better understanding, each vehicle is assigned an earth-fixed frame $X_{E} O_{E} Y_{E}$ and a body-fixed frame $X_{B} O_{B} Y_{B}$ , and their geometric relationship is depicted in Figure 1.

Figure 1.

The earth-fixed frame $X_{E} O_{E} Y_{E}$ and body-fixed frame $X_{B} O_{B} Y_{B}$ of the AAV.

Since this paper primarily investigates the cooperative formation task of multiple AAVs, the focus is placed on position control in the outer loop. Assume that there exist multiple AAV systems composed of N identical individuals, whose dynamics can be described by the following equations (Chen et al., 2023):

\begin{matrix} {\begin{matrix} ẍ_{i} = - \frac{k_{ix}}{m_{i}} ẋ_{i} + \frac{d_{ix}}{m_{i}} + \frac{T_{i}}{m_{i}} (\cos ϕ_{i} \sin θ_{i} \cos ψ_{i} + \sin ϕ_{i} \sin ψ_{i}), \\ ÿ_{i} = - \frac{k_{iy}}{m_{i}} ẏ_{i} + \frac{d_{iy}}{m_{i}} + \frac{T_{i}}{m_{i}} (\cos ϕ_{i} \sin θ_{i} \sin ψ_{i} - \sin ϕ_{i} \cos ψ_{i}), \\ {\overset{··}{z}}_{i} = - \frac{k_{iz}}{m_{i}} ż_{i} + \frac{T_{i}}{m_{i}} \cos ϕ_{i} \cos θ_{i} - g + \frac{d_{iz}}{m_{i}}, \end{matrix} \end{matrix}

(1)

where $x_{i}$ , $y_{i}$ , and $z_{i}$ denote the positions of the mass center of the AAV in $X_{E} O_{E} Y_{E}$ ; $k_{ix}$ , $k_{iy}$ , and $k_{iz}$ represent the aerodynamic damping coefficients; $ϕ_{i}$ , $θ_{i}$ , and $ψ_{i}$ are the roll, pitch, and yaw angles, respectively; $m_{i}$ , g, and $T_{i}$ denote the mass of AAV, the gravitational acceleration, and the total lift, respectively; and $d_{ix}$ , $d_{iy}$ , and $d_{iz}$ stand for the external forces related to modeling uncertainties and disturbances.

Define $η_{i} = {[x_{i}, y_{i}, z_{i}]}^{T}$ , $ϖ_{i} = {\overset{\cdot}{η}}_{i}$ , $K_{i} = {[k_{ix}, k_{iy}, k_{iz}]}^{T}$ , and ${\tilde{τ}}_{d, i} (η_{i}, ϖ_{i}) = {[d_{ix}, d_{iy}, d_{iz}]}^{T}$ , and then the dynamics model is given by

\begin{matrix} {\begin{matrix} {\overset{\cdot}{η}}_{i} = ϖ_{i}, \\ m_{i} {\overset{\cdot}{ϖ}}_{i} = - K_{i} v_{i} - m_{i} gR + u_{i} + {\tilde{τ}}_{d, i} (η_{i}, ϖ_{i}), \end{matrix} \end{matrix}

(2)

where $R = {[0, 0, 1]}^{T}$ and $u_{i} = {[u_{i 1}, u_{i 2}, u_{i 3}]}^{T}$ with $u_{i 1} = T_{i} (\cos ϕ_{i} \sin θ_{i} \cos ψ_{i} + \sin ϕ_{i} \sin ψ_{i})$ , $u_{i 2} = T_{i} (\cos ϕ_{i} \sin θ_{i} \sin ψ_{i} - \sin ϕ_{i} \cos ψ_{i})$ , and $u_{i 3} = T_{i} \cos ϕ_{i} \cos θ_{i}$ , being the position control input. The position control input $u_{i}$ generates the desired angles serving as reference trajectories in the attitude loop, as explained in Dong et al. (2023).

The leaders trajectory in the $X_{E} O_{E} Y_{E}$ coordinate frame can be expressed as:

\begin{matrix} {\begin{matrix} {\overset{\cdot}{η}}_{0, l} = ϖ_{0, l}, \\ {\overset{\cdot}{ϖ}}_{0, l} = μ (η_{0, l}, ϖ_{0, l}), \end{matrix} \end{matrix}

(3)

where $η_{0, l} = {[x_{0, l}, y_{0, l}, z_{0, l}]}^{T}$ , $ϖ_{0, l} = {\overset{\cdot}{η}}_{0, l}$ , and $μ (η_{0, l}, ϖ_{0, l}) = {[μ_{x_{0, l}}, μ_{y_{0, l}}, μ_{ψ_{0, l}}]}^{T}$ . In detail, $[x_{0, l}, y_{0, l}, z_{0, l}]$ represent the leaders position, and $μ (η_{0, l}, ϖ_{0, l})$ represent the acceleration vector of the leaders in the $X_{E} O_{E} Y_{E}$ reference frame.

Assumption 2. The accelerations of the leaders are assumed to be bounded such that $∥ μ (η_{0, l}, ϖ_{0, l}) ∥ \leq β^{*}$ , where $β^{*} > 0$ is a constant.

Problem formulation

This paper focuses on developing cooperative algorithms to address the predefined-time formation tracking control problem of AAVs, which will be formally defined later.

Definition 1. The control objective for AAV formation tracking is considered successfully achieved provided that a predefined time $T > 0$ exists, satisfying the following condition:

\begin{matrix} {\begin{matrix} \lim_{t \to T} ∥ η_{i} - η_{0} - h_{i} (t) ∥ = 0, \\ ∥ η_{i} - η_{0} - h_{i} (t) ∥ = 0, \forall t \geq T, \\ \lim_{t \to T} ∥ ϖ_{i} - ϖ_{0} - \overset{\cdot}{h_{i}} (t) ∥ = 0, \\ ∥ ϖ_{i} - ϖ_{0} - \overset{\cdot}{h_{i}} (t) ∥ = 0, \forall t \geq T, \end{matrix} \end{matrix}

(4)

where $h_{i} (t)$ denotes the formation vector of vehicle i.

Main results

Preliminary of prescribed performance

To ensure that the tracking error $ê_{ηi}$ of the nonlinear system remains within a prescribed bound and achieves steady-state performance within a predefined time, the error is constrained as

\begin{matrix} - ζ_{i} (t) < ê_{ηi} < ζ_{i} (t) \end{matrix}

(5)

where $ζ_{i} (t)$ is a time-varying performance function, and $ê_{ηi}$ will be formally defined in the subsequent section.

A feasible predefined-time performance function is defined as

\begin{matrix} ζ_{i} (t) = {\begin{matrix} (ζ_{i 0} - ζ_{i∞}) \cos^{a_{i}} (\frac{πt}{2 T^{⋆}}) + ζ_{i∞}, & 0 \leq t < T^{⋆} \\ ζ_{i∞}, & t \geq T^{⋆} \end{matrix} \end{matrix}

(6)

where $0 < ζ_{i∞} < | ê_{ηi} (0) | < ζ_{i 0}$ , and $ζ_{i} (t)$ converges from initial error bound $ζ_{i 0}$ to the steady-state bound $ζ_{i∞}$ within a user-defined convergence time $T_{d}$ . The constant $a_{i} > 0$ adjusts the convergence rate.

To transform the constrained error into an unconstrained form, we define

\begin{matrix} T (ê_{ηi}^{*}) = \frac{2}{π} \arctan (ê_{ηi}^{*}) \end{matrix}

(7)

where $ê_{ηi}^{*}$ is the transformed error. This function satisfies:

$T (ê_{ηi}^{*})$ is smooth and strictly increasing;

$T (0) = 0$ , $\lim_{ϖ_{j} \to \pm \infty} T (ê_{ηi}^{*}) = \pm 1$ ;

$- 1 < T (ê_{ηi}^{*}) < 1$ for all $ê_{ηi}^{*} \in ℝ$ .

Consequently, the original tracking error $ê_{ηi}$ can be represented as:

\begin{matrix} ê_{ηi} = ζ_{i} (t) T (ê_{ηi}^{*}) \end{matrix}

(8)

This transformation allows the constrained tracking objective to be equivalently enforced by controlling the unconstrained transformed variable $ê_{ηi}^{*}$ .

Combining (3) and (4), the unconstrained error can be represented as

\begin{matrix} ê_{ηi}^{*} = \tan (\frac{π}{2} \frac{ê_{ηi}}{ζ_{i}}) \end{matrix}

(9)

and its time derivative can be deduced as

\begin{matrix} {\overset{\cdot}{ê}}_{ηi}^{*} = \frac{π}{2} \sec^{2} (\frac{π ê_{ηi}}{2 ζ_{j}}) \frac{{\overset{\cdot}{ê}}_{ηi} ζ_{i} - ê_{ηi} {\overset{\cdot}{ζ}}_{i}}{ζ_{i}^{2}} = ξ_{i} {\overset{\cdot}{ê}}_{ηi} - ξ_{i} {\bar{h}}_{i} \end{matrix}

(10)

in which $ξ_{i} = (\frac{π}{2 ζ_{i}}) \sec^{2} (\frac{π ê_{ηi}}{2 ζ_{i}})$ and ${\bar{h}}_{i} = (ê_{ηi} {\overset{\cdot}{ζ}}_{i} ∕ ζ_{i})$ .

Moreover, taking the time derivative of ${\overset{\cdot}{ê}}_{ηi}^{*}$ yields the second-order dynamics of the transformed error:

\begin{matrix} {\overset{··}{ê}}_{ηi}^{*} = ξ_{i} {\overset{··}{ê}}_{ηi} - ξ_{i} {\overset{\cdot}{\bar{h}}}_{i} + {\overset{\cdot}{ξ}}_{i} ({\overset{\cdot}{ê}}_{ηi} - {\bar{h}}_{i}) \end{matrix}

(11)

Hierarchical control algorithm for predefined-time formation tracking

This subsection develops a PTHC method to address the TVFT problem of AAVs. The proposed approach consists of two main components: first, a distributed estimator is designed to allow each vehicle to accurately obtain the leaders state information; second, an RL-based controller is introduced to ensure that the formation task is accomplished within a user-specified time horizon. The structure and implementation process of this control strategy are described in detail below.

Define ${\hat{η}}_{i}$ and ${\hat{ϖ}}_{i}$ as the estimators of $η_{i}$ and $ϖ_{i}$ , respectively. It then follows that

\begin{matrix} {\tilde{η}}_{i} = {\hat{η}}_{i} - η_{0, l}, \\ {\tilde{ϖ}}_{i} = {\hat{ϖ}}_{i} - ϖ_{0, l}, \\ δ_{η, i} = \sum_{j \in V_{i}} a_{ij} ({\hat{η}}_{i} - {\hat{η}}_{j}) + b_{i} ({\hat{η}}_{i} - η_{0, l}), \\ δ_{ϖ, i} = \sum_{j \in V_{i}} a_{ij} ({\hat{ϖ}}_{i} - {\hat{ϖ}}_{j}) + b_{i} ({\hat{ϖ}}_{i} - ϖ_{0, l}), \\ ê_{ηi} = η_{i} - {\hat{η}}_{i} - h_{i} (t), \\ ê_{ϖi} = {\overset{\cdot}{ê}}_{ηi} = ϖ_{i} - {\hat{ϖ}}_{i} - ḣ_{i} (t), \\ e_{ηi} = η_{i} - η_{0, l} - h_{i} (t), \\ e_{ϖi} = ė_{ηi} = ϖ_{i} - ϖ_{0, l} - ḣ_{i} (t) . \end{matrix}

The sliding surfaces are given by:

\begin{matrix} \begin{matrix} ϕ_{i} = δ_{ϖ, i} + (μ_{1} + μ_{2} ψ_{2} (t)) δ_{η, i} . \end{matrix} \end{matrix}

(12)

\begin{matrix} {\begin{matrix} ŝ_{i, 1} = ê_{ηi}^{*}, \\ ŝ_{i, 2} = {\overset{\cdot}{ŝ}}_{i, 1} + Θ (ŝ_{i, 1}) \end{matrix} \end{matrix}

(13)

where $Θ (ŝ_{i, 1}) = (α_{1}^{⋆} + \frac{ρ_{1}^{⋆} β_{1}^{⋆}}{T_{s}} ψ_{ℓ}^{\frac{1}{β_{1}^{⋆}}} (t)) ŝ_{i, 1}, β_{1}^{⋆} \in [2, \infty), ρ_{1} \geq \frac{1}{λ_{2} (L_{B})},$ where $T_{s}$ is the predefined-time parameter.

The dynamics of the proposed predefined-time distributed estimator are governed by:

\begin{matrix} {\begin{matrix} {\overset{\cdot}{\hat{η}}}_{i} & = {\hat{ϖ}}_{i}, \\ {\overset{\cdot}{\hat{ϖ}}}_{i} & = {\hat{μ}}_{i}, \\ {\hat{μ}}_{i} & = {(b_{i} + \sum_{j \in N_{i}} a_{ij})}^{- 1} {b_{i} μ_{0, r} + \sum_{j \in N_{i}} (a_{ij} {\hat{μ}}_{j}) \\ - (μ_{1} + μ_{2} ψ_{2} (t)) {\overset{\cdot}{δ}}_{η, i} - K_{f} sgn (ϕ_{i}) \\ - (ε_{1} + ε_{2} ψ_{1} (t)) ϕ_{i}}, \end{matrix} \end{matrix}

(14)

Inspired by Zhao et al. (2025), the predefined-time local controller is designed as follows:

\begin{matrix} {\begin{matrix} τ_{i}^{a} = & Λ_{i}^{- 1} (τ_{i 1} + τ_{i 2}), \\ τ_{i 1} = & - Ṙ (ψ_{i}) v_{i} + {\overset{\cdot}{\hat{ϖ}}}_{i} + ḧ_{i} - ξ_{i}^{- 1} \overset{\cdot}{Θ} (ŝ_{i, 1}) - Λ_{i}^{2} {\hat{τ}}_{d, i} \\ + Λ_{i} (K_{i} v_{i} + m_{i} gR) + {\overset{\cdot}{\bar{h}}}_{i} \\ - ξ_{i}^{- 1} {\overset{\cdot}{ξ}}_{i} ({\overset{\cdot}{ê}}_{ηi} - {\bar{h}}_{i}), \\ τ_{i 2} = & - Φ (ŝ_{i, 2}) . \end{matrix} \end{matrix}

(15)

where $Λ_{i} = m_{i}^{- 1}$ , and ${\hat{τ}}_{d, i}$ denotes the estimated coupling disturbance vector ${\tilde{τ}}_{d, i} (η_{i}, ϖ_{i})$ . The auxiliary function is defined as

\begin{matrix} Φ (ŝ_{i, 1}) = (α_{3} + \frac{β_{2}^{⋆}}{T_{r}} ψ_{ℓ}^{\frac{1}{β_{2}^{⋆}}} (t)) ŝ_{i, 2} - k_{i} sgn (ŝ_{i, 1}), \end{matrix}

(16)

with $β_{2}^{⋆} \in [2, \infty)$ and the predefined-time parameter $T_{r} \in ℝ_{+}$ .

The proposed hierarchical control scheme integrates two layers: a predefined-time distributed estimator and a predefined-time local controller. The overall architecture is depicted in Figure 2.

Figure 2.

Structure of the proposed hierarchical control method.

It can be approximated by an actor NN as

\begin{matrix} {\hat{τ}}_{d, i} = ŵ_{a}^{T} Φ_{c} (z_{i}), \end{matrix}

(17)

where $ŵ_{a}$ is the adaptive weight vector, and $z_{i} = {[x_{i}, v_{i}]}^{T}$ represents the input. The error is given by

\begin{matrix} μ_{ai} (t) = {\tilde{w}}_{a}^{T} Φ_{c} (z_{i}), \end{matrix}

(18)

where ${\tilde{w}}_{a} = ŵ_{a} - w_{a}^{*} .$ Furthermore, define

\begin{matrix} δ_{ai} (t) = μ_{ai} (t) + k_{I} [{\hat{F}}_{i} (t) - I_{id} (t)], \end{matrix}

(19)

where $I_{id} = 0$ denotes the ideal cost, and $k_{I} > 0$ is the gain constant. Defining the performance index as

\begin{matrix} E_{ai} = \frac{1}{2} δ_{ai} {(t)}^{T} δ_{ai} (t), \end{matrix}

(20)

the adaptive updating law is thereby obtained as

\begin{matrix} \begin{matrix} {\overset{\cdot}{ŵ}}_{a} & = - θ_{a} \frac{\partial E_{ai}}{\partial δ_{ai}} \frac{\partial δ_{ai} (t)}{\partial μ_{ai}} \frac{\partial μ_{ai}}{\partial ŵ_{a}} \\ = - θ_{a} (μ_{ai} (t) + k_{I} {\hat{F}}_{i} (t)) Φ_{c} (z_{i}) \end{matrix} \end{matrix}

(21)

where $θ_{a} > 0$ , and $μ_{ai} (t)$ denotes an uncertain function. Equation (21) can rewritten as

\begin{matrix} {\overset{\cdot}{ŵ}}_{a} = - θ_{a} [ŵ_{a}^{T} Φ_{c} (z_{i}) + k_{I} {\hat{F}}_{i} (t)] Φ_{c} (z_{i}) . \end{matrix}

(22)

Define the cost function as

\begin{matrix} F_{i} (t) = \int_{t}^{\infty} \exp (- \frac{κ}{β} - t) φ_{i} (κ) dκ, \end{matrix}

(23)

where $β > 0$ denotes the discount factor associated with future costs. The instantaneous cost is defined as $φ_{i} (t) = {\bar{η}}_{i}^{T} V {\bar{η}}_{i} + {\bar{ϖ}}_{i}^{T} M {\bar{ϖ}}_{i},$ with ${\bar{η}}_{i} = η_{i} - {\hat{η}}_{i}$ and ${\bar{ϖ}}_{i} = ϖ_{i} - {\hat{ϖ}}_{i}$ . $V$ and $M$ are positive diagonal matrices. To evaluate the cost function using the actor-critic framework, the augmented local error state vector is defined as $E_{i} = {[{\bar{η}}_{i}^{T}, {\bar{ϖ}}_{i}^{T}]}^{T}$ . The objective is to minimize $F_{i} (t)$ , which can be approximated by exploiting the universal approximation capability of NNs as

\begin{matrix} F_{i} (t) = w_{c}^{* T} Φ_{c} (E_{i}) + ϵ_{ci} . \end{matrix}

(24)

Here, $ϵ_{ci} > 0$ , and $w_{c}^{*}$ is a weight matrix. Design ${\hat{F}}_{i} (t) = ŵ_{c}^{T} Φ_{c} (E_{i}),$ where $ŵ_{c}$ is the estimated weight vector corresponding to $w_{c}^{*}$ . The temporal-difference error is defined as

\begin{matrix} δ_{i} (t) = φ_{i} (t) - \frac{1}{α} {\hat{F}}_{i} (t) + {\overset{\cdot}{\hat{F}}}_{i} (t), \end{matrix}

(25)

which is employed to guide the parameter adaptation of the critic. Accordingly, the weight update law is formulated as ${\overset{\cdot}{ŵ}}_{c} = - θ_{c} \frac{\partial E_{ci}}{\partial ŵ_{c}},$ where $θ_{c} > 0$ is the learning rate, and $E_{ci} = \frac{1}{2} δ_{i} {(t)}^{T} δ_{i} (t)$ denotes the instantaneous cost associated with the critic. Substituting the above definitions yields

\begin{matrix} {\overset{\cdot}{ŵ}}_{c} = - θ_{c} δ_{i} (t) [- \frac{1}{α} \frac{\partial {\hat{F}}_{i} (t)}{\partial ŵ_{c}} + \frac{\partial}{\partial ŵ_{c}} (\frac{\partial {\hat{F}}_{i} (t)}{\partial {\bar{η}}_{i}}) {\overset{\cdot}{\bar{η}}}_{i}] . \end{matrix}

(26)

By further simplification, the critic weight adaptation law can be expressed as

\begin{matrix} {\overset{\cdot}{ŵ}}_{c} = - θ_{c} (φ_{i} (t) + ŵ_{c}^{T} Ψ_{i}) Ψ_{i}, \end{matrix}

(27)

where $Ψ_{i} = - \frac{Φ_{c} (E_{i})}{α} + \nabla Φ_{c} (E_{i}) \overset{\cdot}{E_{i}}$ . ■

Analysis of the distributed estimator layer

The stability of the predefined-time distributed estimator in equation (14) is analyzed in this section. To facilitate the analysis, we introduce the following compact vector forms: $\hat{η} = {({\hat{η}}_{1}, {\hat{η}}_{2}, \dots, {\hat{η}}_{N})}^{T}$ , $\hat{ϖ} = {({\hat{ϖ}}_{1}, {\hat{ϖ}}_{2}, \dots, {\hat{ϖ}}_{N})}^{T}$ , $ẽ_{η} = {(ẽ_{η 1}, ẽ_{η 2}, \dots, ẽ_{ηN})}^{T}$ , $ẽ_{ϖ} = {(ẽ_{ϖ 1}, ẽ_{ϖ 2}, \dots, ẽ_{ϖN})}^{T}$ , $\hat{μ} = {({\hat{μ}}_{1}, {\hat{μ}}_{2}, \dots, {\hat{μ}}_{N})}^{T}$ , and $s = {(s_{1}, s_{2}, \dots, s_{N})}^{T}$ . The matrix $Π$ is defined as $Π = (L + B) \otimes I_{3}$ .

Theorem 1. Given that Assumption 1 is satisfied, if $T_{1} > 0$ , $T_{2} > 0$ , and $0 < E_{2} < E_{1} < 2 E_{2}$ hold, then the error variables ${\tilde{η}}_{i}$ and ${\tilde{ϖ}}_{i}$ converge to zero for all $i \in V$ within $t \leq T_{f} = T_{1} + T_{2}$ .

Proof. First, we examine the estimator layer. Considering ${\tilde{η}}_{i}$ and ${\tilde{ϖ}}_{i}$ , equation (14) can be transformed into a compact form as follows:

\begin{matrix} {\begin{matrix} \overset{\cdot}{\tilde{η}} = \tilde{ϖ}, \\ \overset{\cdot}{\tilde{ϖ}} = \hat{μ} - ν, \end{matrix} \end{matrix}

(28)

where $ν = 1_{N} \otimes μ_{0, r}$ . To facilitate the analysis of ${\hat{μ}}_{i}$ in equation (14), we can use an alternative representation: ${\hat{μ}}_{i} = {(\sum_{j \in N_{i}} a_{ij} + b_{i})}^{- 1} (y_{i} (t) + \sum_{j \in N_{i}} a_{ij} {\hat{μ}}_{j})$ , with $y_{i} (t)$ representing the remainder of the original formula. From this, we can derive $\sum_{j \in N_{i}} a_{ij} ({\hat{μ}}_{i} - {\hat{μ}}_{j}) + b_{i} {\hat{μ}}_{i} = y_{i} (t)$ , leading to $Π \hat{μ} = y (t)$ where $y (t) = {(y_{1} (t), y_{2} (t), \dots, y_{N} (t))}^{T}$ . Consequently, equation (14) can be reformulated into the following compact form:

\begin{matrix} {\begin{matrix} \overset{\cdot}{\hat{η}} & = \hat{ϖ}, \\ \overset{\cdot}{\hat{ϖ}} & = \hat{μ}, \\ \hat{μ} & = Π^{- 1} {(B \otimes I_{3}) ν - (μ_{1} + μ_{2} ψ_{2} (t)) Π \overset{\cdot}{\tilde{η}} \\ - (ε_{1} + ε_{2} ψ_{1} (t)) ϕ} \end{matrix} \end{matrix}

(29)

where $ϕ_{i} = {(ϕ_{1}, ϕ_{2}, \dots, ϕ_{N})}^{T}$ . In addition, by incorporating equation (12), we can express the time derivative of ϕ as follows:

\begin{matrix} \begin{matrix} {\overset{\cdot}{ϕ}}_{i} = - K_{f} sgn (ϕ_{i}) - (ε_{1} + ε_{2} ψ_{1} (t)) ϕ_{i} . \end{matrix} \end{matrix}

(30)

A Lyapunov function for equation (12) is selected as $V_{1} = \frac{1}{2 ε_{2}} ϕ_{i}^{T} ϕ_{i} .$ It then follows that

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{1} (ϕ_{i}) & \leq - \frac{1}{ε_{2}} (ε_{1} + ε_{2} ψ_{1} (t)) ϕ_{i}^{T} ϕ_{i} \\ = - ε_{1} V_{1} (ϕ_{i}) - 2 ψ_{1} (t) V_{1} (ϕ_{i}) . \end{matrix} \end{matrix}

(31)

By applying Lemma 1, it becomes evident that $ϕ_{i}$ reaches zero within the bounded interval $t \leq T_{1}$ . Once $ϕ_{i} = 0$ is derived, one has

\begin{matrix} δ_{ϖ, i} = - (μ_{1} + μ_{2} ψ_{2} (t)) δ_{η, i} . \end{matrix}

(32)

Then select the following function as the candidate Lyapunov function: $V_{2} (δ_{η, i}) = \frac{1}{2 μ_{2}} {(δ_{η, i})}^{T} δ_{η, i} .$ The derivative of $V_{2} (δ_{η, i})$ is computed as

\begin{matrix} {\overset{\cdot}{V}}_{2} (δ_{η, i}) \leq - 2 μ_{1} V_{2} (δ_{η, i}) - 2 ψ_{2} (t) V_{2} (δ_{η, i}), \end{matrix}

(33)

which implies that the designed $δ_{η, i}$ and $δ_{ϖ, i}$ converge to origin in predefined time $T_{f}$ . It can be further obtained that $\lim_{t \to T_{f}} ∥ {\tilde{η}}_{i} ∥ = 0$ and $\lim_{t \to T_{f}} ∥ {\tilde{ϖ}}_{i} ∥ = 0$ , and for $t \geq T_{f}$ , $∥ {\tilde{η}}_{i} ∥ = 0$ and $∥ {\tilde{ϖ}}_{i} ∥ = 0$ for all $i \in V$ . In summary, the predefined-time distributed estimator algorithm (14) can achieve stability within the predefined time $T_{f}$ . ■

Prescribed-performance predefined-time RL controller design for AAVs

Following the conclusion of Theorem 1, the sufficient condition for guaranteeing predefined-time formation tracking is derived in this subsection.

Theorem 2. With the local controller (15) and updating laws (22) to (27), if $α_{3} \geq \frac{1}{2}$ and $θ_{c} ζ_{1}^{2} \geq 2 θ_{a} k_{I}^{2} ζ_{2}^{2}$ , then the actor and critic NN weights are uniformly ultimately bounded (UUB), and the AAVs achieve prescribed-performance predefined-time formation tracking within $T^{⋆} = T_{f} + T_{r} + T_{s}$ .

Proof. Define the Lyapunov function as $V_{3} = V_{c} + V_{a} + V_{2}$ , where $V_{c} = \frac{1}{2} {\tilde{w}}_{c}^{T} {\tilde{w}}_{c}, V_{a} = \frac{1}{2} {\tilde{w}}_{a}^{T} {\tilde{w}}_{a}, V_{2} = \frac{1}{2} ŝ_{i, 2}^{T} ŝ_{i, 2}$ , ${\tilde{w}}_{c} = ŵ_{c} - w_{c}^{*}, {\tilde{w}}_{a} = ŵ_{a} - w_{a}^{*}$ . Then differentiating $V_{c}$ by equation (22): ${\overset{\cdot}{V}}_{c} = {\tilde{w}}_{c}^{T} {\overset{\cdot}{ŵ}}_{c} = - θ_{c} {\tilde{w}}_{c}^{T} (φ_{i} (t) + ŵ_{c}^{T} Ψ_{i}) Ψ_{i} .$ Since $δ_{i} \leq ℏ$ with $ℏ > 0$ , $δ_{i} (t) = φ_{i} (t) - \frac{1}{α} {\hat{F}}_{i} (t) + {\overset{\cdot}{\hat{F}}}_{i} (t)$ and ${\hat{F}}_{i} (t) = ŵ_{c}^{T} Φ_{c} ({\bar{η}}_{i})$ , therefore:

\begin{matrix} \begin{matrix} φ_{i} (t) \leq & \frac{w_{c}^{* T} Φ_{c} ({\bar{η}}_{i})}{α} + \frac{ε_{c}}{α} + ℏ \\ - \nabla (w_{c}^{* T} Φ_{c} ({\bar{η}}_{i}) + ε_{c}) {\overset{\cdot}{Φ}}_{c} ({\bar{η}}_{i}) \\ \leq & - w_{c}^{* T} Ψ_{i} + ϵ_{ci}, \end{matrix} \end{matrix}

(34)

where $ϵ_{ci} = ε_{ci} ∕ α + \nabla ε_{ci} {\overset{\cdot}{Φ}}_{c} ({\bar{η}}_{i}) + ℏ .$ Therefore:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{c} & \leq - θ_{c} {\tilde{w}}_{c}^{T} ϵ_{ci} Ψ_{i} - θ_{c} {\tilde{w}}_{c}^{T} {\tilde{w}}_{c} Ψ_{i}^{T} Ψ_{i} \\ \leq - \frac{θ_{c}}{2} Ψ_{i}^{T} Ψ_{i} {\tilde{w}}_{c}^{T} {\tilde{w}}_{c} + \frac{θ_{c}}{2} ϵ_{ci}^{T} ϵ_{ci} . \end{matrix} \end{matrix}

(35)

The derivative of $V_{a}$ from equation (27) is:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{a} \leq & - θ_{a} {\tilde{w}}_{a}^{T} k_{I} {\hat{F}}_{i} (t) Φ_{c} (z_{i}) - θ_{a} {\tilde{w}}_{a}^{T} Φ_{c} (z_{i}) w_{a}^{* T} Φ_{c} (z_{i}) . \end{matrix} \end{matrix}

(36)

Applying Young’s inequality gives:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{a} \leq & - \frac{1}{2} ∥ {\tilde{w}}_{a} ∥^{2} ∥ Φ_{c} (z_{i}) ∥^{2} + \frac{θ_{a}^{2}}{2} ∥ w_{a}^{*} ∥^{2} ∥ Φ_{c} (z_{i}) ∥^{2} \\ + θ_{a} k_{I}^{2} (∥ {\tilde{w}}_{c} ∥^{2} ∥ Φ_{c} ({\bar{η}}_{i}) ∥^{2} + ∥ w_{c}^{*} ∥^{2} ∥ Φ_{c} ({\bar{η}}_{i}) ∥^{2}) \end{matrix} \end{matrix}

(37)

The derivative of $ŝ_{i, 2}$ is given by

\begin{matrix} \begin{matrix} {\overset{\cdot}{ŝ}}_{i, 2} = & {\overset{··}{ê}}_{ηi}^{*} + \overset{\cdot}{Θ} (ŝ_{i, 1}) \\ = & ξ_{i} ({\overset{\cdot}{ϖ}}_{i} - {\overset{\cdot}{\hat{ϖ}}}_{i} - ḧ_{i} (t)) - ξ_{i} {\overset{\cdot}{\bar{h}}}_{i} + {\overset{\cdot}{ξ}}_{j} ({\overset{\cdot}{ê}}_{ηi} - {\bar{h}}_{i}) \\ + \overset{\cdot}{Θ} (ŝ_{i, 1}) \end{matrix} \end{matrix}

(38)

Substituting equation (15) into equation (38):

\begin{matrix} \begin{matrix} {\overset{\cdot}{ŝ}}_{i, 2} = Λ_{i} {\tilde{τ}}_{d, i} - Λ_{i} {\hat{τ}}_{d, i} - ξ_{i} Φ (ŝ_{i, 2}) \end{matrix} \end{matrix}

(39)

Using the NN properties, it follows that $Λ_{i} {\tilde{τ}}_{d, i} - Λ_{i} {\hat{τ}}_{d, i} = Λ_{i} ε_{bi}$ , where $ε_{bi} > 0$ .

Consequently, it can be derived that:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{2} = & ŝ_{i, 2}^{T} {\overset{\cdot}{ŝ}}_{i, 2} \\ \leq & ŝ_{i, 2}^{T} (Λ_{i} ε_{bi} - ξ_{i} Φ (ŝ_{i, 2})) \\ = & - ŝ_{i, 2}^{T} ξ_{i} Φ (ŝ_{i, 2}) + ŝ_{i, 2} Λ_{i} ε_{bi} \\ \leq & - 2 α_{3} V_{2} - 2 \frac{β_{2}^{⋆}}{T_{r}} μ^{\frac{1}{β_{2}^{⋆}}} (t) ξ_{i} V_{2} - (k_{i} - | ε_{bi} |) | ŝ_{i, 2} | | Λ_{i} | \\ \leq & - 2 α_{3} V_{2} \end{matrix} \end{matrix}

(40)

where $k_{i} > | ε_{bi} |$ . Combining equations (35), (37), and (40), it can be obtained:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{3} \leq & - 2 α_{3} V_{2} + \frac{θ_{a}^{2}}{2} ∥ w_{a}^{*} ∥^{2} ∥ Φ_{c} (z_{i}) ∥^{2} \\ - \frac{θ_{c} Ψ_{i}^{T} Ψ_{i} - 2 θ_{a} k_{I}^{2} ∥ Φ_{c} ({\bar{η}}_{i}) ∥^{2}}{2} ∥ {\tilde{w}}_{c} ∥^{2} \\ - \frac{∥ Φ_{c} (z_{i}) ∥^{2} ∥ {\tilde{w}}_{a} ∥^{2}}{2} + \frac{θ_{c}}{2} ϵ_{ci}^{T} ϵ_{ci} \\ + θ_{a} k_{I}^{2} ∥ w_{c}^{*} ∥^{2} ∥ Φ_{c} ({\bar{η}}_{i}) ∥^{2} . \end{matrix} \end{matrix}

(41)

Let $θ_{c} ζ_{1}^{2} \geq 2 θ_{a} k_{I}^{2} ζ_{2}^{2}$ . Then:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{3} \leq - a_{1} V_{3} + b_{1} \end{matrix} \end{matrix}

(42)

where $a_{1} = \min {ζ_{3}^{2}, θ_{c} ζ_{1}^{2} - 2 θ_{a} k_{I}^{2} ζ_{2}^{2}, 2 α_{3} - 1}$ ; $b_{1} = θ_{c} ϵ_{ci}^{T} ϵ_{ci} ∕ 2 + θ_{a} k_{I}^{2} ∥ w_{c}^{*} ∥^{2} ∥ Φ_{c} ({\bar{η}}_{i}) ∥^{2} + θ_{a} ∥ w_{a}^{*} ∥^{2} ∥ Φ_{c} (z_{i}) ∥^{2} ∕ 2$ ; and $∥ Ψ_{i} ∥ \leq ζ_{1}$ , $∥ Φ_{c} ({\bar{η}}_{i}) ∥ \leq ζ_{2}$ , and $∥ Φ_{c} (z_{i}) ∥ \leq ζ_{3}$ , where $ζ_{1}, ζ_{2}, ζ_{3} > 0$ . It follows that $V_{3} \leq V_{3} (0) + b_{1} ∕ a_{1}$ , indicating that $V_{3}$ is UUB, and thus, $s_{i}$ , ${\tilde{w}}_{c}$ , ${\tilde{w}}_{a}$ , $ŵ_{c}$ , and $ŵ_{a}$ are also UUB.

Next, it is proved that the states reach the origin within the predefined time. The derivative of the Lyapunov function $V_{2}$ is given by:

\begin{matrix} \begin{matrix} {\overset{\cdot}{V}}_{2} & \leq - (2 α_{3} - 1) V_{2} - 2 \frac{β_{2}^{⋆}}{T_{r}} ψ_{r}^{\frac{1}{β_{2}^{⋆}}} (t) V_{2} \end{matrix} \end{matrix}

(43)

By Lemma 1, $ŝ_{i, 2} \to 0_{n}$ as $t \to T_{r}$ , implying that the sliding mode manifold is reached within the predefined time $T_{r}$ . When $ŝ_{i, 2} = 0_{n}$ , the dynamics of $ŝ_{i, 0}$ reduce to

\begin{matrix} {\overset{\cdot}{ŝ}}_{i, 0} = - (α_{1}^{⋆} + \frac{ρ_{i}^{⋆} β_{i}^{⋆}}{T_{s}} ψ_{s}^{\frac{1}{β_{i}^{⋆}}} (t)) ŝ_{i, 0} . \end{matrix}

(44)

According to Lemma 1, the construction of the Lyapunov candidate $V = \frac{1}{2} ŝ_{i, 0}^{T} ŝ_{i, 0}$ guarantees that the state $ŝ_{i, 0}$ will be driven to the origin in a time not exceeding $T_{r} + T_{s}$ . Since $ŝ_{i, 0} = ê_{ηi}^{*}$ , equation (10) further guarantees that both $ê_{ηi}$ and its derivative $ê_{ϖi}$ vanish within the same predefined time $T_{r} + T_{s}$ .

Furthermore, since both $ê_{ηi}$ and $ê_{ϖi}$ converge to the origin within $T^{⋆}$ , and ${\tilde{η}}_{i}$ and ${\tilde{ϖ}}_{i}$ vanish simultaneously, it follows from the triangular inequality that

\lim_{t \to T^{⋆}} ∥ e_{ηi} ∥ \leq \lim_{t \to T^{⋆}} ∥ ê_{ηi} ∥ + \lim_{t \to T^{⋆}} ∥ {\tilde{η}}_{i} ∥ = 0,

and

\lim_{t \to T^{⋆}} ∥ e_{ϖi} ∥ \leq \lim_{t \to T^{⋆}} ∥ ê_{ϖi} ∥ + \lim_{t \to T^{⋆}} ∥ {\tilde{ϖ}}_{i} ∥ = 0 .

Therefore, both $e_{ηi}$ and $e_{ϖi}$ remain zero for all $t \geq T^{⋆}$ , which ensures that the predefined-time formation tracking problem of AAVs is solved. ■

Remark 1. Theoretically, the predefined-time parameter T can be specified as any positive constant regardless of the initial conditions. However, in practical AAV applications, the selection of T is fundamentally constrained by the actuator saturation limits. From the control laws (15) and (16), it can be observed that the control gain is inversely proportional to T; thus, a smaller T yields a faster convergence rate but necessitates a larger control input $u_{i}$ . To avoid actuator saturation and potential system instability, T should be selected based on the AAV’s physical capabilities, such as the maximum rotor thrust. In practice, a feasible T is typically determined by calculating the maximum required control effort for a given set of initial errors, ensuring that the control signals remain within the saturation bounds $∥ u_{i} ∥ \leq u_{\max}$ .

Remark 2. Although the proposed control scheme integrates NNs, RL, and sliding mode control, the computational burden is carefully managed to ensure compatibility with the limited onboard resources of small-scale AAVs. Specifically, the hierarchical architecture decomposes the complex formation problem into an upper-layer distributed estimator and a lower-layer local controller, avoiding high-dimensional integrated optimization. The RL component employs a shallow radial basis function NN, which significantly reduces the backpropagation complexity compared to deep learning architectures. Furthermore, the nonsingular sliding mode control provides an analytical control law, enabling high-frequency execution with minimal latency. Since each vehicle only processes local states and information from immediate neighbors, the computational complexity for each AAV is $O (1)$ relative to the total number of vehicles, ensuring excellent scalability for large-scale swarms.

Remark 3. It should be noted that the stability analysis in Theorems 1 and 2 is established under the assumption of a static communication topology. However, real-world AAV operations frequently encounter intermittent connectivity. To rigorously extend the proposed PTHC framework to dynamic communication links, the network should be modeled as a Markovian switching topology governed by a continuous-time Markov process. Fully incorporating these stochastic switching dynamics introduces significant mathematical complexity that falls outside the scope of the current study. Therefore, extending the current predefined-time stability criteria to Markovian switching topologies to address intermittent connectivity is identified as an important direction for our immediate future work.

Simulations

In this section, quadrotor AAVs are employed as examples to verify the effectiveness of the proposed scheme.

The communication topology is shown in Figure 3, where the nodes $α_{1}$ and $α_{2}$ serve as the leaders of subgroups $V_{1} = 1, 2, 3$ and $V_{2} = 4, 5, 6$ , respectively. The corresponding Laplacian matrix is given by:

L = [\begin{matrix} 1 & - 1 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & - 1 & 0 & 1 \\ 0 & - 3 & 3 & 0 & 0 & 0 \\ 0 & 0 & 0 & 3 & 0 & - 3 \\ 0 & 0 & 0 & - 1 & 1 & 0 \\ 0 & 0 & 0 & - 3 & - 1 & 4 \end{matrix}],

and $B = diag {3, 5, 0, 8, 2, 0}$ . The time-varying formation offsets for AAVs are expressed as $h_{i} = [\frac{1}{2} \cos (\frac{iπ}{2} + t), \frac{1}{2} \sin (\frac{iπ}{2} + t), 0]^{T}$ for $V_{1}$ , and $h_{i} = [\frac{1}{2} \cos ((\frac{2 iπ}{3} - \frac{π}{6}) + t), \frac{1}{2} \sin ((\frac{2 iπ}{3} - \frac{π}{6}) + t), 0]^{T}$ for $V_{2}$ . Accordingly, the trajectories of the leaders $α_{1}$ and $α_{2}$ are given by $η_{0, 1} = {[6.6 \cos (0.3 t), - 1 + 6.6 \sin (0.3 t), - 0.3 t]}^{T}$ and $η_{0, 2} = {[2.6 \cos (t), - 2 + 2.6 \sin (t), - 1.5 t]}^{T}$ . Moreover, the specific physical parameters of the AAV are available in the study by Jin et al. (2024) for reference.

Figure 3.

The communication topology of the AAVs.

The prescribed-performance parameters in equation (6) are selected as $ζ_{i 0} = 10$ , $ζ_{i∞} = 0.05$ , and $a_{i} = 10$ . The initial values of ${\hat{η}}_{i}$ , ${\hat{ϖ}}_{i}$ , $η_{i}$ , and $ϖ_{i}$ at $t = 0$ are chosen arbitrarily. Other controller parameters are set to $t_{0} = 0 s$ , $α_{1} = 20$ , $α_{2} = 12$ , and $α_{3} = 1$ . In the actor-critic framework, both networks start with random outputs. The actor produces random control inputs, and the critic gives initial evaluations that are gradually refined through iterations to enhance performance. The learning rates are chosen as $δ_{a} = 0.2$ and $δ_{c} = 0.5$ .

The simulation results of the AAV formation tracking control problem are shown in Figures 4 –8, with the analysis summarized below.

Figure 4 shows the estimated states ${\hat{η}}_{i}$ and ${\hat{ϖ}}_{i}$ , which closely track the leaders time-varying states.

Figure 5 illustrates how the state deviations $η_{i} - h_{i}$ and $ϖ_{i} - h_{i}$ evolve over time. Because these deviations continuously adapt to the formation maneuvers, their trajectories do not settle at constant values. The tracking error responses are plotted in Figure 6, demonstrating that all errors stay confined within the prescribed performance limits. Furthermore, the results indicate that the tracking errors reach a designated level of precision within the predefined time.

Figure 7 shows the AAV formation trajectories, confirming effective tracking and successful formation of the desired geometry. As illustrated in the figure, the AAVs maintain a high level of cooperative consistency during convergence, which verifies the stability and robustness of the proposed control scheme throughout the formation process. This also demonstrates the algorithms’ applicability to tasks like aerial patrol, swarm delivery, and search-and-rescue operations.

Figure 8 shows the time responses of the actor and critic weight derivatives for vehicle 1. At the beginning, both weights exhibit rapid adaptive changes, followed by a gradual stabilization phase, indicating that the proposed algorithm possesses effective self-tuning and stable control performance. Fifteen distinct curves are presented, each showing the weight changes of the vehicles’ actor and critic networks.

Figure 4.

Subplots (a)–(c) depict the evolution of ${\hat{η}}_{i}$ , while (d)–(f) illustrate the evolution of ${\hat{ϖ}}_{i}$ .

Figure 5.

Subplots (a)–(c) display the evolution of $η_{i} - h_{i}$ , while (d)–(f) show the evolution of $ϖ_{i} - ḣ_{i}$ .

Figure 6.

Pictures (a)–(c) and (d)–(f) present the evolutions of $ê_{ηi}$ and $ê_{ϖi}$ .

Figure 7.

Using equations (14) and (15), the two-dimensional time-varying formation tracking control (TVFTC) trajectories of the AAVs are depicted in the $X_{E} O_{E} Y_{E}$ space.

Figure 8.

The evolution process of actor-critic weights ${\overset{\cdot}{ŵ}}_{a}$ and ${\overset{\cdot}{ŵ}}_{c}$ for vehicle 1.

Algorithm 1. Simulation Procedure of the Proposed Algorithm
Require: Allocation
1: The initial values of $η_{i}$ , $ϖ_{i}$ , ${\hat{η}}_{i}$ , ${\hat{ϖ}}_{i}$ , ${\hat{w}}_{c, i}$ , ${\hat{w}}_{a, i}$ are given;
2: Set $η_{0}^{1}$ , $η_{0}^{2}$ , $h_{i}$ ;
3: Set parameters $δ_{a}$ , $δ_{c}$ , $k_{I}$ , $α$ , $ρ$ , $t_{0}$ , $T_{2}$ , $T_{p}$ , $a$ , $b$ , $α_{1}$ , $α_{2}$ , $α_{3}$ ;
4: Set simulation step size $d t$ and maximum running time $T_{m a x}$ ;
5: for $t = 0$ to $T_{m a x}$ with step $d t$ do
6: Step 1: Running the distributed resilient estimator;
7: for $i = 1 : N$ do
8: Receive relative information from neighbor sets $j \in N_{i}$ ;
9: Build upon (14), calculate ${\hat{η}}_{i}$ , ${\hat{ϖ}}_{i}$ ;
10: end for
11: Step 2: RL local controller and weight adaptation;
12: for $i = 1 : N$ do
13: Compute errors $\bar{η_{i}} = η_{i} - {\hat{η}}_{i}$ , $\bar{ϖ_{i}} = ϖ_{i} - {\hat{ϖ}}_{i}$ , and sliding surface $s_{i}$ ;
14: Compute control input $τ_{i}^{a}$ via (15) using current actor weights ${\hat{w}}_{a, i}$ ;
15: Apply $τ_{i}^{a}$ to vehicle $i$ and evaluate instantaneous cost $φ_{i} (t)$ ;
16: Update critic weights ${\hat{w}}_{c, i}$ via (27);
17: Update actor weights ${\hat{w}}_{a, i}$ via (22) based on the updated critic;
18: end for
19: end for
Ensure: Return $η_{i}$ , $ϖ_{i}$ ;

To further validate the superior performance and robustness of the PTHC framework, a comparative simulation study is conducted. Under identical initial conditions and time-varying external disturbances, the proposed PTHC scheme is comprehensively compared with a traditional fixed-time sliding mode control method. As demonstrated by the tracking performance curves of the traditional method in Figure 9 and the subsequent quantitative analysis (taking the first vehicle as an example), the mean square error (MSE) of the tracking performance under the PTHC scheme is significantly reduced compared to the traditional approach in Table 1. This fully substantiates that under complex external disturbances, the PTHC framework can achieve higher-precision trajectory tracking and exhibits superior anti-disturbance robustness.

Figure 9.

Pictures (a)–(c) and (d)–(f) present the evolutions of $ê_{ηi}$ and $ê_{ϖi}$ by adopting the algorithm used in Han et al. (2021).

Table 1.

Quantitative comparison of MSE under external disturbances (Han et al., 2023).

State variables	Traditional method MSE	Proposed method MSE
$ê_{ηi} (1)$ ( $m^{2}$ )	7.678051	17.831361
$ê_{ηi} (2)$ ( $m^{2}$ )	0.770555	19.711658
$ê_{ηi} (3)$ ( ${rad}^{2}$ )	1.343835	5.381250
$ê_{ϖi} (1)$ ( ${(m/s)}^{2}$ )	0.784405	3.724332
$ê_{ϖi} (2)$ ( ${(m/s)}^{2}$ )	1.990891	2.421950
$ê_{ϖi} (3)$ ( ${(rad/s)}^{2}$ )	0.952454	1.275401

Conclusion

The predefined-time formation tracking problem of AAVs under uncertain conditions is investigated in this paper. A two-layer hierarchical control framework is proposed. The upper-layer distributed estimator is designed based on the nonsingular sliding mode control, which eliminates the singularity issues of traditional sliding mode approaches and enables accurate estimation of the leaders trajectory within a predefined time. The lower-layer local controller is developed using RL to achieve the TVFT objective. Future work will focus on integrating deep RL with multi-agent game-theoretic mechanisms to realize intelligent evolution of control strategies, autonomous coordination, and attack recognition, thereby enhancing the systems’ autonomy and intelligence in dynamic mission environments.

Footnotes

ORCID iDs

Tao Han

Bo Xiao

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was funded by the National Natural Science Foundation of China (62473136) and the Hubei Provincial Education Department Science and Technology Research Program for Young Talents (Q20232502).

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

Data sharing is not applicable to this article as no datasets were generated or analysed during the current study.

References

Chen

Liang

Pan

, et al. (2023) Human-in-the-loop consensus tracking control for UAV systems via an improved prescribed performance approach. IEEE Transactions on Aerospace and Electronic Systems 59(6): 8380–8391.

Ding

Zhang

Miao

, et al. (2024) Event-based finite-time formation tracking control for UAV with bearing measurements. IEEE Transactions on Industrial Electronics 72(7): 7482–7492.

Dong

Chai

Yao

, et al. (2023) Terminal sliding mode attitude tracking control for unmanned vehicle with predefined-time stability. Aerospace Science and Technology 142: 108669.

Gao

Jin

, et al. (2024a) Impulsive bounded consensus of two-layer multi-agent networks under deception attacks. Transactions of the Institute of Measurement and Control 46(10): 1979–1990.

Gao

Wang

Xia

, et al. (2024b) Fixed-time neuroadaptive backstepping tracking control for uncertain nonlinear systems with predictor based learning. IEEE Transactions on Automation Science and Engineering 22: 3147–3159.

Gong

Yang

(2024) Secure tracking control via fixed-time convergent reinforcement learning for a UAV CPS. IEEE/CAA Journal of Automatica Sinica 11(7): 1699–1701.

Greene

Bell

Nivison

, et al. (2023) Deep neural network-based approximate optimal tracking for unknown nonlinear systems. IEEE Transactions on Automatic Control 68(5): 3171–3177.

(2025) Fully distributed adaptive consensus tracking for uncertain multi-agent systems. Transactions of the Institute of Measurement and Control 47(6): 1117–1125.

Guan

Yiu

KFC

, et al. (2025) Predefined-time and predefined-accuracy sliding mode control with unknown bound uncertainties. IEEE Transactions on Circuits and Systems I: Regular Papers 72: 7380–7388.

10.

Han

Chi

Guan

, et al. (2017a) Distributed three-dimensional formation containment control of multiple unmanned aerial vehicle systems. Asian Journal of Control 19(3): 1103–1113.

11.

Han

Guan

Chi

, et al. (2017b) Multi-formation control of nonlinear leader-following multi-agent systems. ISA Transactions 69: 140–147.

12.

Han

Guan

, et al. (2016) Three-dimensional containment control for multiple unmanned aerial vehicles. Journal of the Franklin Institute 353(13): 2929–2942.

13.

Han

Guan

Xiao

, et al. (2021) Bipartite average tracking for multi-agent systems with disturbances: Finite-time and fixed-time convergence. IEEE Transactions on Circuits and Systems I: Regular Papers 68(10): 4393–4402.

14.

Jia

Liu

(2023) Predefined-time nonsingular sliding mode control and its application to nonlinear systems. IEEE Transactions on Industrial Informatics 20(4): 5829–5837.

15.

Jin

Ahn

Xiang

(2024) Adaptive distributed fuzzy control for prescribed-time formation of multiple autonomous aerial vehicles. IEEE Transactions on Aerospace and Electronic Systems 61(2): 1589–1599.

16.

Wang

, et al. (2025) Terminal constraint guidance and predefined-time event-triggered consensus control for multi-UAV aerial rendezvous. Nonlinear Dynamics 113(13): 16695.

17.

Ding

Zheng

, et al. (2023) NN-based reinforcement learning optimal control for inequality-constrained nonlinear discrete-time systems with disturbances. IEEE Transactions on Neural Networks and Learning Systems 35(11): 15507–15516.

18.

Liu

, et al. (2025) Consensus tracking control for nonlinear second-order multi-agent systems with input magnitude and rate constraints. Transactions of the Institute of Measurement and Control 47(9): 1849–1858.

19.

Liu

Dang

, et al. (2023) Deep reinforcement learning of UAV tracking control under wind disturbances environments. IEEE Transactions on Instrumentation and Measurement 72: 1–13.

20.

Meng

Zhang

Jiang

(2023) Fixed-time optimal fault-tolerant formation control with prescribed performance for fixed-wing UAVs under dual faults. IEEE Transactions on Signal and Information Processing over Networks 9: 875–887.

21.

Miao

Zhang

Jiang

(2024) Fixed-time collision-free fault-tolerant formation control of multi-UAVs under actuator faults. IEEE Transactions on Cybernetics 54(6): 3679–3691.

22.

Ren

Zhou

, et al. (2021) Prescribed-time cluster lag consensus control for second-order non-linear leader-following multiagent systems. ISA Transactions 109: 49–60.

23.

Shao

Zheng

(2022) Predefined-time sliding mode control with prescribed convergent region. IEEE/CAA Journal of Automatica Sinica 9(5): 934–936.

24.

Song

Fan

, et al. (2024) Predefined-time sliding mode attitude control for liquid-filled spacecraft with large amplitude sloshing. European Journal of Control 77: 100970.

25.

Wang

, et al. (2023) Safe reinforcement learning and adaptive optimal control with applications to obstacle avoidance problem. IEEE Transactions on Automation Science and Engineering 21(3): 4599–4612.

26.

Wei

Chen

Shi

(2023) Dynamic event-triggered consensus cost-based switching control for UAV formation with disturbances. IEEE Transactions on Intelligent Vehicles 9(2): 3531–3543.

27.

Ding

, et al. (2023a) Finite-time fault-tolerant formation control for distributed multi-vehicle networks with bearing measurements. IEEE Transactions on Automation Science and Engineering 21(2): 1346–1357.

28.

Liang

Gou

, et al. (2023b) Heterogeneous mission planning for multiple UAV formations via metaheuristic algorithms. IEEE Transactions on Aerospace and Electronic Systems 59(4): 3924–3940.

29.

Yan

Han

, et al. (2023) Event-triggered formation control for time-delayed discrete-time multi-agent system applied to multi-UAV formation flying. Journal of the Franklin Institute 360(5): 3677–3699.

30.

Yang

Pan

Cao

, et al. (2024) Predefined-time fault-tolerant consensus tracking control for multi-UAV systems with prescribed performance and attitude constraints. IEEE Transactions on Aerospace and Electronic Systems 60(4): 4058–4072.

31.

Zhang

Duan

(2024) Multi-AAV formation search using variable resolution cognitive map and tree-type topologies under limited sensing environments. IEEE Transactions on Aerospace and Electronic Systems 61(2): 2734–2751.

32.

Zhang

Wang

(2024) Prescribed performance tracking control of time-delay nonlinear systems with output constraints. IEEE/CAA Journal of Automatica Sinica 11(7): 1557–1565.

33.

Zhao

Han

Xiao

, et al. (2025) Neural network-based reinforcement learning for prescribed-time formation tracking of hybrid networked marine systems under attacks. ISA Transactions 167: 305–318.

34.

Zhao

Yang

(2024) Reinforcement-learning-based fuzzy adaptive finite-time optimal resilient control for large-scale nonlinear systems under false data injection attacks. IEEE Transactions on Fuzzy Systems 32(4): 2483–2495.

35.

Zhou

Chen

Zhang

, et al. (2024) Distributed finite-time prescribed performance for multiple unmanned aerial vehicle with time-varying external disturbance. IEEE Internet of Things Journal 11(9): 16969–16980.