A learning-based generalized dynamic predictive control for PMSM position servo system with uncertain dynamics

Abstract

This paper presents a learning-based generalized dynamic predictive control (GDPC) approach for a permanent magnet synchronous motor (PMSM) position servo system, aiming to achieve intelligent and fine-grained operation under uncertain environments. Considering the limitations of the human-configured generalized predictive control (GPC) scheme with the fixed horizon to cope with diverse operating conditions, the investigated strategy further integrates a deep reinforcement learning (DRL)-based horizon online-updating mechanism. In particular, an extended state observer (ESO) is first constructed for the estimation of lumped disturbance to modify the deviation of the prediction model. The analytical solution of the benchmark GPC algorithm is then obtained by solving an optimization problem of the designed performance index. To optimize the prediction horizon of GPC, a DRL agent is then trained offline. Real-time horizon adjustment is finally implemented on an experimental setup that combines a digital signal processor (DSP) and a Beckhoff controller. A series of simulations and experiments validate the efficacy of the proposed control approach. The proposed GDPC method utilizes a deep deterministic policy gradient (DDPG) algorithm to optimize the prediction horizon in real time, achieving improved control performance under varying operating conditions.

Keywords

generalized dynamic predictive control deep reinforcement learning PMSM position servo system parameter optimization

Introduction

Permanent magnet synchronous motors (PMSMs) play a crucial role as primary components in numerous industrial applications, demanding precise and rapid position servo response to fulfill accuracy and speed prerequisites (Ding et al., 2023; Wang et al., 2025; Xu et al., 2024). The distinct advantage of PMSMs lies in their remarkable control capabilities across a range of applications. However, widespread utilization of PMSMs presents substantial challenges in achieving high-performance control standards, which primarily attributed to uncertainties stemming from parameter perturbations, friction torque, external disturbances, and variations in different operating conditions (Mandra et al., 2019; Tian et al., 2022).

Nowadays, conventional linear control strategies have been extensively employed in motion control systems. However, their capability to meet the precision and adaptability demands of modern industrial applications is often deemed insufficient. This is due to the complexities of PMSMs, which are often characterized by nonlinear behaviors and uncertainties, hindering the establishment of precise models required for traditional control techniques. Consequently, various nonlinear control theories such as model predictive control (MPC) (Cao et al., 2024; Jia et al., 2024; Zhang et al., 2025), active disturbance rejection control (ADRC) (Hou et al., 2023; Yang et al., 2025; Zuo et al., 2021), and sliding mode control (SMC) (Yue et al., 2023; Zhang et al., 2021) have been thoroughly investigated and successfully implemented to the high-performance PMSM control applications. Among these approaches, MPC stands out for its real-time optimization capability, making it a promising method to tackle nonlinear and complex systems while effectively managing multiple constraints (Belda and Vosmik, 2016; Song et al., 2023).

Generalized predictive control (GPC) is considered notable because it effectively handles system constraints and exhibits high robustness by employing a continuous-time model, which avoids the performance degradation associated with sampling period selection in traditional discrete MPC (Yang et al., 2015; Zheng et al., 2021). We analyze how ADRC and SMC focus on disturbance rejection, while GPC provides a more systematic optimization framework for position servo tasks. Deep reinforcement learning (DRL) refers to a class of machine learning algorithms where an agent learns to make decisions by interacting with an environment to maximize cumulative rewards. DRL is employed in this work because it can autonomously learn optimal control policies without requiring explicit system models, making it suitable for complex, nonlinear systems like PMSM drives.

It is knowable that the effectiveness of the standard MPC strategy based on a discrete-time model is significantly influenced by the chosen sampling period in controller design (Yang et al., 2015). Generally speaking, employing a small sampling period alongside a relatively large horizon may result in heightened computational demands, while the extension of the sampling period unavoidably diminishes both the dynamic response and anti-interference capabilities of the system to a certain extent. As an alternative, GPC based on continuous-time models stands out as a notable predictive control method, which not only features robustness, rapid dynamic response, and easy handling of system constraints but also excels in computational efficiency and straightforward engineering implementation (Zheng et al., 2021). Nonetheless, since the traditional GPC relies on nominal system modeling, it encounters challenges when dealing with load disturbances, model parameter drift, and unmodeled dynamics (Yang et al., 2017). Furthermore, employing a fixed control gain poses challenges in attaining optimal control performance across diverse operating conditions (Dong et al., 2024; Zhang et al., 2023). This emphasizes the demands for the predictive control algorithm to exhibit both robustness and adaptability simultaneously. The prediction horizon $T_{p}$ dictates the trade-off between response speed and stability. As $T_{p}$ increases, the control gains $k_{1}$ and $k_{2}$ decrease, leading to slower but more stable response. Conversely, a small $T_{p}$ yields aggressive control but risks instability under disturbances. Fixed-horizon GPC is suboptimal because it cannot adapt to varying operating conditions and disturbances.

As an intelligent algorithm, the DRL possesses the capability of autonomous learning and solving complex problems (Li et al., 2025; Ma et al., 2023). By virtue of this promising characteristic, the DRL algorithm has gained attention in the traditional controller design, such as applying parameter optimization to PI differential controllers, the robust stabilization strategy (Lee et al., 2020), and MPC methodologies (Cui et al., 2024; Pan et al., 2024). In contrast to some DRL methods, such as adaptive dynamic programming (ADP) techniques that generally require specific model information to accurately capture system dynamics (Liu et al., 2021), or Q-learning methods which show greater suitability in discrete action spaces (Liu and Su, 2022), the deep deterministic policy gradient (DDPG) algorithm (Ouyang et al., 2025) stands out as a well-known actor-critic approach appreciated for its effectiveness in handling continuous control tasks; therefore, it is suitable for the data-driven feedback control system (Liu et al., 2024). However, it should be noted that although the above studies have showcased effective control in semi-physical simulations, their utilization in optimizing controller design for physical electrical drive systems remains limited.

Recently, Dong et al. (2024) proposed an adaptive GPC method for PMSM speed regulation, demonstrating effective disturbance rejection under mismatched disturbances. While their work focuses on speed control, our paper addresses the more demanding position servo control problem, which requires higher steady-state accuracy and different dynamic response characteristics. This distinction highlights the need for a control framework capable of handling the additional complexities of position tracking under uncertain dynamics. To address the limitations of fixed-horizon GPC and extend the GPC paradigm from speed regulation to precision position control, this work integrates a DRL-based adaptive horizon tuning mechanism. The DRL agent operates as a high-level parameter optimizer, dynamically adjusting the prediction horizon $T_{p}$ in real time based on system states, thereby enhancing adaptability to varying operating conditions.

Motivated by the above analysis, a DRL-based generalized dynamic predictive control (GDPC) approach is put forward for the PMSM position servo system, which seamlessly integrates the benchmark GPC and DDPG algorithm, along with a variable horizon. To implement the proposed strategy, the robust GPC is first obtained from the rotor position prediction model calibrated by an extended state observer (ESO), which is deployed in a digital signal processor (DSP). Meanwhile, the DRL training results are introduced into the horizon adjustment of GPC, and the agent is further deployed in a Beckhoff controller. Compared with the existing results, the main contributions of this paper are summarized as follows:

The proposed DRL-based GDPC method incorporates a real-time horizon adjustment mechanism, achieving an order-of-magnitude improvement in computational efficiency while maintaining comparable control performance. This enables intelligent control for PMSMs under a wide range of operating conditions.

Compared with existing observer-based robust GPC, the proposed method not only improves adaptability to diverse operating conditions via DRL-based horizon tuning, but also enhances disturbance rejection under simultaneous load torque and inertia variations, as evidenced by reduced position fluctuation.

Different from existing semi-physical simulation validations for DRL-based control algorithms, the synergy between DSP and Beckhoff controllers delivers a practical dual-processor architecture that decouples optimization from real-time control, facilitating reliable deployment in real power electronic applications without additional hardware complexity.

The remainder of this paper is organized as follows: Problem formulation is presented in section “Problem formulation.” Section “DRL-based GDPC design” provides a detailed design process of the proposed GDPC method. Simulation and experimental tests under various scenarios are introduced in sections “Simulation verification” and “Experimental verification,” respectively. Section “Conclusion” summarizes this article. Finally, the rigorous stability analysis is provided in Appendix 1.

Problem formulation

This section presents a typical physical model of the PMSM servo system, followed by an outline of the control objective.

PMSM servo model

The dynamic equations of a PMSM servo system are generally expressed as Yan et al. (2018)

\begin{matrix} \frac{d θ_{m}}{dt} & = w_{m} \end{matrix}

(1a)

\begin{matrix} J \frac{d ω_{m}}{dt} & = \frac{3 n_{p} ψ_{f}}{2} i_{q} - B_{v} w_{m} - T_{L} + Δ T \end{matrix}

(1b)

\begin{matrix} L_{s} \frac{d i_{q}}{dt} & = u_{q} - R_{s} i_{q} - n_{p} w_{m} L_{s} i_{d} - n_{p} w_{m} ψ_{f} \end{matrix}

(1c)

\begin{matrix} L_{s} \frac{d i_{d}}{dt} & = u_{d} - R_{s} i_{d} + n_{p} w_{m} L_{s} i_{q} \end{matrix}

(1d)

where $i_{d}, i_{q}$ are the $d, q$ axis stator currents; $u_{d}, u_{q}$ are the $d, q$ axis stator voltages; $L_{s}$ is the stator inductance; $R_{s}$ is the stator resistance; $n_{p}$ is the number of pole pairs; $ψ_{f}$ is the flux generated by the permanent magnet; $w_{m}$ is the angular velocity; $θ_{m}$ is the rotor position; J is the moment of inertia; $T_{L}$ is the external load torque; $B_{v}$ is the system viscous friction coefficient; and $Δ T$ refers to the torque disturbance caused by model perturbations.

To proceed, the stator current $i_{q}$ and the rotor position $θ_{m}$ are correspondingly selected as the system input and output. Consequently, the motion equations (1a)–(1b) can be reorganized as the following compact form

\begin{matrix} {\begin{matrix} \frac{d θ_{m}}{dt} & = w_{m} \\ \frac{d w_{m}}{dt} & = b_{0} u + d (t) \end{matrix} \end{matrix}

(2)

where $b_{0} = \frac{K_{t}}{J}$ is the system parameter, $K_{t} = \frac{3}{2} n_{p} ψ_{f}$ is the torque constant, $u = {i_{q}}^{*}$ is the reference current, and $d (t)$ is the lumped disturbance caused by the current loop tracking error, the uncertain load torque, the friction torque, and so on, which can be specifically expressed by

\begin{matrix} \begin{matrix} d (t) = - \frac{1}{J} [K_{t} (i_{q}^{*} - i_{q}) + B_{v} w_{m} + T_{L} - Δ T] \end{matrix} \end{matrix}

(3)

Starting from the PMSM mechanical dynamics and differentiating the position gives the second-order relationship. Defining $u = i_{q}^{*}$ and $b_{0} = K_{t} ∕ J$ , we obtain the compact form. The term $d (t)$ explicitly includes current tracking error, friction, load torque, and parameter uncertainties. The assumption $i_{q} \approx i_{q}^{*}$ is based on time-scale separation. In high-performance PMSM drives, the current loop (electrical dynamics) is designed with bandwidth ≈ 1 kHz, while the position loop (mechanical dynamics) operates at ≈ 100 Hz. Thus, $i_{q}$ tracks $i_{q}^{*}$ almost instantaneously relative to mechanical motion. Any residual error $(i_{q} - i_{q}^{*})$ is absorbed into $d (t)$ and compensated by the observer.

From the perspective of industrial application, it is reasonable to make the following assumption.

Assumption 1. The derivative of the lumped disturbance $d (t)$ is a bounded signal, satisfying $| ḋ (t) | \leq C$ , where $C \in ℝ_{+}$ .

The assumption $| ḋ (t) | \leq C$ is standard in disturbance observer literature. Physically, load torque and friction cannot change infinitely fast due to finite actuator power and mechanical inertia. This boundedness ensures the observer estimation error remains bounded.

In practical applications, PMSM servo systems frequently operate in uncertain environments, which involve perturbations in load torque $T_{L}$ , moment of inertia J, and unmodeled dynamics such as nonlinear friction. Under such conditions, the control objective is to achieve “fine-grained operation,” that is, the ability to maintain high tracking precision and smooth dynamic transitions despite these uncertainties.

Control objective

Considering the application requirements of the position servo system, the control objectives of this article can be concisely summarized as follows:

Precise position control: The primary objective is to attain precise position control for the PMSM servo system. This is achieved through the implementation of a GPC approach, ensuring that the rotor position $θ_{m} (t)$ closely follows the reference position $θ_{r} (t)$ .

Adaptation to operating conditions: The control system is designed to exhibit adaptability to various operating conditions and environmental factors, even in the presence of perturbations and uncertainties. To this end, a DRL strategy is introduced to facilitate parameter online updating, which can be expressed as

\begin{matrix} J (μ) = E_{μ} [\sum_{t = 1}^{\infty} γ^{t - 1} {| θ_{m} (t) - θ_{r} (t) |}^{2}] \end{matrix}

(4)

where μ is the policy determined by the DRL agent and $γ \in [0, 1]$ is the discount factor.

DRL-based GDPC design

This section presents the main procedure for designing the DRL-based GPC approach, which includes the design of the benchmark GPC and the horizon tuning mechanism.

Benchmark GPC design

Position prediction model construction

To proceed, the rotor position $θ_{m} (t + τ)$ can be predicted within a period of τ via Taylor series expansion as follows

\begin{matrix} θ_{m} (t + τ) \approx θ_{m} (t) + τ {\overset{\cdot}{θ}}_{m} (t) + \dots + \frac{τ^{2 + r}}{(2 + r)!} θ_{m}^{(2 + r)} (t) \end{matrix}

(5)

To facilitate the implementation on DSP, the control order r is set to zero. Consequently, (5) can be simplified into

\begin{matrix} \begin{matrix} θ_{m} (t + τ) = T (τ) [U (t) + X (t)] \end{matrix} \end{matrix}

(6)

where $T (τ) = [1, τ, \frac{τ^{2}}{2!}]$ , $U (t) = {[0, 0, b_{0} u (t)]}^{⊤}$ , $X (t) = {[θ_{m} (t), {\overset{\cdot}{θ}}_{m} (t), d (t)]}^{⊤}$ .

Using Taylor series expansion, the future position can be approximated as

\begin{matrix} θ_{m} (t + τ) \approx θ_{m} (t) + τ {\overset{\cdot}{θ}}_{m} (t) + \frac{τ^{2}}{2!} {\overset{··}{θ}}_{m} (t) \end{matrix}

(7)

setting control order $r = 0$ assumes constant control over $[t, t + T_{p}]$ . This simplification enables the analytical solution with computation time of only $8.9 μ s$ , crucial for real-time implementation.

It is apparent from (6) that the rotor angular velocity ${\overset{\cdot}{θ}}_{m} (t)$ and the lumped disturbance $d (t)$ are unavailable states. With this in mind, observer techniques can be used for the soft-sensing of undetectable signals.

Unknown signal observation

First, by defining $θ_{m} = x_{1}$ and ${\overset{\cdot}{θ}}_{m} = x_{2}$ , the state-space model can be formulated as

\begin{matrix} {\begin{matrix} ẋ_{1} = x_{2} \\ ẋ_{2} = b_{0} u + d (t) \end{matrix} \end{matrix}

(8)

For system (8), an ESO is employed to estimate the angular velocity and disturbance simultaneously as follows

\begin{matrix} {\begin{matrix} {\overset{\cdot}{Φ}}_{1} = Φ_{2} - κ_{1} (Φ_{1} - x_{1}) \\ {\overset{\cdot}{Φ}}_{2} = Φ_{3} - κ_{2} (Φ_{1} - x_{1}) + b_{0} u \\ {\overset{\cdot}{Φ}}_{3} = - κ_{3} (Φ_{1} - x_{1}) \end{matrix} \end{matrix}

(9)

where $x_{1}$ , $x_{2}$ , and $x_{3}$ represent the rotor position, the angular velocity, and the total disturbance, that is, $θ_{m}$ , $ω_{m}$ , and $d (t)$ , respectively; $Φ_{1}$ , $Φ_{2}$ , and $Φ_{3}$ denote the estimates of $x_{1}$ , $x_{2}$ , and $x_{3}$ from the ESO, respectively. $κ_{1}, κ_{2}$ , and $κ_{3}$ are the positive design gains of the observer. The ESO follows the fundamental principles of ADRC originally proposed by Han (2009).

The ESO gains are designed using the bandwidth parameterization technique, which places all observer poles at $- δ$ (with $δ > 0$ ). By setting the characteristic polynomial of $H$ as ${(s + δ)}^{3}$ , we obtain

\begin{matrix} s^{3} + κ_{1} s^{2} + κ_{2} s + κ_{3} = {(s + δ)}^{3} = s^{3} + 3 δ s^{2} + 3 δ^{2} s + δ^{3} \end{matrix}

which yields $κ_{1} = 3 δ$ , $κ_{2} = 3 δ^{2}$ , $κ_{3} = δ^{3}$ . This parameterization ensures that the observer error dynamics are Hurwitz and exponentially stable with a tunable convergence rate. The bandwidth δ determines the estimation speed: a larger δ leads to faster convergence but increases sensitivity to measurement noise, while a smaller δ provides smoother estimates at the cost of slower response. In practice, δ is selected based on a trade-off between these factors and the system’s sampling capabilities. For our PMSM setup, after extensive experimental tuning, we chose $δ = 800 rad/s$ , which provides a good balance between fast disturbance estimation and noise immunity.

From system (8) and observer (9), the estimation error dynamics system can be calculated as

\begin{matrix} ė = H e + D ḋ \end{matrix}

(10)

where $e = {[e_{1}, e_{2}, e_{3}]}^{⊤}, e_{i} = Φ_{i} - x_{i} (i = 1, 2, 3)$ is the estimation error, and $x_{3} = d$ . $H = [\begin{matrix} - κ_{1} & 1 & 0 \\ - κ_{2} & 0 & 1 \\ - κ_{3} & 0 & 0 \end{matrix}]$ and $D = {[0, 0, - 1]}^{⊤}$ .

It can be observed from (10) that since $ḋ \neq 0$ in general, the error system does not achieve asymptotic convergence to zero. However, this does not compromise disturbance rejection capability. The system satisfies the stronger property of input-to-state stability (ISS) (Khalil, 2002), which guarantees that the estimation error remains bounded and its ultimate bound is proportional to the magnitude of ḋ. Specifically, from ISS theory, we have

\begin{matrix} ∥ e (t) ∥ \leq α e^{- βt} ∥ e (0) ∥ + γ \sup_{τ \in [0, t]} | ḋ (τ) | \end{matrix}

(11)

where $α, β > 0$ and γ is the ISS gain. For the designed ESO with bandwidth $δ = 800$ rad/s, the gain γ is approximately 0.0012, which is very small. Consequently, even under time-varying disturbances, the steady-state estimation error is negligible.

Receding-horizon optimization

For the purpose of achieving the optimal control of the PMSM servo system, the performance index is defined as

\begin{matrix} J_{p} = \frac{1}{2} \int_{0}^{T_{p}} [{({\hat{θ}}_{m} (t + τ) - {\hat{θ}}_{r} (t + τ))}^{2} + p {(û (t) - û_{r} (t))}^{2}] dτ \end{matrix}

(12)

where $T_{p}$ is the horizon, ${\hat{θ}}_{r}$ represents the expected position, û and $û_{r}$ are control input and its steady-state signal, and $p \geq 0$ is a weighted coefficient. All variables are denoted as

\begin{matrix} \begin{matrix} {\hat{θ}}_{m} (t + τ) = T (τ) [U (t) + \hat{X} (t)] \\ {\hat{θ}}_{r} (t + τ) = T (τ) Y_{r} (t), \hat{X} (t) = {[θ_{m} (t), {\hat{\overset{\cdot}{θ}}}_{m} (t), \hat{d} (t)]}^{⊤} \\ Y_{r} (t) = {[θ_{r} (t), {\overset{\cdot}{θ}}_{r} (t), {\overset{··}{θ}}_{r} (t)]}^{⊤}, û_{r} (t) = \frac{{\overset{··}{θ}}_{r} (t) - \hat{d} (t)}{b_{0}} \end{matrix} \end{matrix}

(13)

The current optimal control input is calculated according to the defined performance index at each sampling time. Therefore, substituting (13) into (12) yields

\begin{matrix} J_{p} & = \frac{1}{2} \int_{0}^{T_{p}} [{(T (τ) (\hat{X} + U - Y_{r}))}^{2} + p {(û - û_{r})}^{2}] dτ \\ = \frac{1}{2} ({\hat{X}}^{⊤} + U^{⊤} - {Y_{r}}^{⊤}) \bar{T} (\hat{X} + U - Y_{r}) + \frac{1}{2} p T_{p} {(û - û_{r})}^{2} \end{matrix}

(14)

where $\bar{T} = \int_{0}^{T_{p}} T^{⊤} (τ) T (τ) dτ$ .

Separate the matrix $\bar{T}$ into a submatrix form, that is, $\bar{T} = [\begin{matrix} {\bar{T}}_{1} & {\bar{T}}_{2} \\ {\bar{T}}_{2}^{⊤} & {\bar{T}}_{3} \end{matrix}]$ , where ${\bar{T}}_{1} \in ℝ^{2 \times 2}$ , ${\bar{T}}_{2} \in ℝ^{2 \times 1}$ , ${\bar{T}}_{3} \in ℝ$ . Then, taking the partial derivative of $J_{p}$ with respect to û, one has

\begin{matrix} \frac{\partial J_{p}}{∂û} = b_{0} [{\bar{T}}_{2}^{⊤}, {\bar{T}}_{3} + \frac{p T_{p}}{{b_{0}}^{2}}] (\hat{X} - Y_{r}) + ({b_{0}}^{2} {\bar{T}}_{3} + p T_{p}) û \end{matrix}

(15)

From the cost function $J_{p} = \frac{1}{2} \int_{0}^{T_{p}} [{({\hat{θ}}_{m} (t + τ) - {\hat{θ}}_{r} (t + τ))}^{2} + p {(û (t) - û_{r} (t))}^{2}] dτ$ , taking $\partial J_{p} ∕ ∂û = 0$ yields the optimal control law.

Receding-horizon optimization

Furthermore, the optimal control law $u^{*}$ can be derived by letting $\partial J_{p} ∕ ∂û = 0$ , namely,

\begin{matrix} \begin{matrix} u^{*} & = - \frac{1}{b_{0}} [{({\bar{T}}_{3} + \frac{p T_{p}}{{b_{0}}^{2}})}^{- 1} {\bar{T}}_{2}^{⊤}, 1] (\hat{X} - Y_{r}) \\ = - \frac{1}{b_{0}} [k_{1} (θ_{m} (t) - θ_{r} (t)) + k_{2} ({\hat{\overset{\cdot}{θ}}}_{m} (t) - {\overset{\cdot}{θ}}_{r} (t))] \\ + \frac{1}{b_{0}} [{\overset{··}{θ}}_{r} (t) - \hat{d} (t)] \end{matrix} \end{matrix}

(16)

where $k_{1}, k_{2}$ determined by the first row of the matrix ${({\bar{T}}_{3} + \frac{p T_{p}}{{b_{0}}^{2}})}^{- 1} {\bar{T}}_{2}^{⊤}$ are $k_{1} = \frac{10 b_{0}^{2} T_{p}^{2}}{3 b_{0}^{2} T_{p}^{4} + 60 p}$ and $k_{2} = \frac{5 b_{0}^{2} T_{p}^{3}}{2 b_{0}^{2} T_{p}^{4} + 40 p}$ .

Note that the dimensional consistency of the gains can be verified as follows: $b_{0}$ has units rad/(A⋅s²), $T_{p}$ is in seconds, and p is dimensionless. Therefore, $k_{1}$ has units $s^{- 2}$ (appropriate for position error feedback) and $k_{2}$ has units $s^{- 1}$ (appropriate for velocity error feedback). The expressions are thus dimensionally consistent.

The GPC control law in (16) fundamentally differs from a conventional PID controller in several key aspects. First, the gains $k_{1}$ and $k_{2}$ are not tuned heuristically but are analytically derived from the minimization of a receding-horizon cost function, as shown above. This ensures optimality with respect to the defined performance index. Second, the controller minimizes a cost function that penalizes the predicted future error $\int_{0}^{T_{p}} {({\hat{θ}}_{m} (t + τ) - {\hat{θ}}_{r} (t + τ))}^{2} dτ$ , giving it a predictive nature absent in PID control. Third, the gains $k_{1}$ and $k_{2}$ vary dynamically with the prediction horizon $T_{p}$ , which itself is optimized online by the DRL agent. This hierarchical adaptation mechanism provides enhanced flexibility and performance under varying operating conditions.

Remark 1. As shown in (16), unlike the general state feedback controller, the control gains $k_{1}$ and $k_{2}$ in the obtained GPC control law is a function of the horizon $T_{p}$ and the control input weighted factor p. In general, $T_{p}$ governs the response characteristics of the system, whereas p enables constraints to be placed on the control energy. However, in many cases, these parameters are traditionally fine-tuned based on engineering experience. Given the complexity of the industrial application scenario, the selected predictive horizon may be too conservative for broad operating conditions. From this perspective, an online-updating mechanism for the variable horizon is expected to be designed.The parameter p weights control effort in the cost function and significantly affects closed-loop poles. $p = 0.01$ was chosen based on motor current limits to prevent saturation while maintaining performance.

Remark 2. Increasing the horizon to enhance control performance is balanced by the decrease in accuracy of the prediction along the time window. The Taylor expansion prediction becomes less accurate as $τ \to T_{p}$ due to neglected higher-order terms. The DRL agent learns to balance this trade-off, selecting $T_{p}$ that minimizes the cost function while ensuring stability.

DRL-based variable horizon of GPC

In this subsection, we enhance the classical GPC by integrating the DDPG-based learning algorithm for the horizon tuning mechanism design. The detailed design framework of the proposed DRL-based GDPC is depicted in Figure 1. The system operates as follows: the encoder measures rotor position $θ_{m}$ , which is used by the ESO to estimate the angular velocity ${\hat{\overset{\cdot}{θ}}}_{m}$ and the lumped disturbance $\hat{d}$ . The GPC, utilizing ${\hat{\overset{\cdot}{θ}}}_{m}$ , $\hat{d}$ , and the reference trajectory $(θ_{r}, {\overset{\cdot}{θ}}_{r}, {\overset{··}{θ}}_{r})$ , minimizes a cost function over the prediction horizon $T_{p}$ to compute the optimal current reference $i_{q}^{*}$ . The current loop, consisting of Clarke/Park transformations and PI controllers, regulates the actual $d, q -axis$ currents $i_{d}, i_{q}$ to track $i_{d}^{*} = 0$ and $i_{q}^{*}$ , generating voltage commands $u_{d}^{*}, u_{q}^{*}$ which are transformed back to three-phase voltages $u_{a}, u_{b}, u_{c}$ to drive the PMSM. Meanwhile, the DRL agent observes the system state $S_{t} = {θ_{m}, θ_{m_del}, ε, ε_del}$ and receives a reward $r_{t}$ based on tracking error, then outputs an updated prediction horizon $T_{p}$ to the GPC. This closed-loop interaction enables adaptive horizon tuning in real time without explicit hardware-level separation.

Figure 1.

Block diagram of the proposed DRL-based GDPC for PMSM servo system, illustrating the interaction among the ESO, GPC optimizer, current control loop, and DRL agent for adaptive horizon tuning.

The DRL (DDPG) agent is integrated as a high-level “parameter optimizer” that dynamically selects $T_{p}$ based on real-time system states. Stability is mathematically guaranteed by Theorem 1 in Appendix 1, which proves the system remains uniformly bounded when $T_{p} \in [0.02, 0.1] s$ . The DRL agent’s action space is constrained to this stable range, ensuring theoretical rigor while leveraging DRL’s adaptability.

The specific design philosophy of the state space, the action space, the reward function, and the actor and critic network are illustrated in the following steps. Furthermore, Figure 2 presents a diagram illustrating the DRL-based optimization procedure for the variable horizon.

Figure 2.

DRL-based optimization procedure for the variable horizon.

State space

Considering the control objective of ensuring highly precise position tracking with smooth and rapid dynamic response, the fundamental state of the position servo system is characterized by the output position $θ_{m}$ , the tracking error $ε = θ_{m} - θ_{r}$ , and their delayed signals $θ_{m_del} (t)$ and $ε_{_del} (t)$ . Consequently, the state space is formed by

\begin{matrix} S_{t} = {θ_{m} (t), θ_{m_del} (t), ε (t), ε_{_del} (t)} \end{matrix}

(17)

Although ${\overset{\cdot}{θ}}_{m}$ is not explicitly included, $θ_{m_del}$ and $ε_del$ provide temporal information that implicitly captures velocity and acceleration trends. This avoids noise amplification from numerical differentiation while maintaining observability of the system’s dynamic state.

Action space

It is worth clarifying that the horizon $T_{p}$ and control effort weight p largely impact the control performance. Specifically, the horizon $T_{p}$ determines the response performance of the position servo system, while the weight p determines the penalty on the control energy. Therefore, when p is human-configured as a fixed value via the “trial and error” manner, $T_{p}$ can be dynamically chosen by the action derived from the DRL algorithm. Furthermore, if the range of action is not constrained, the agent may explore unstable control strategies in real systems, leading to system instability or poor performance. Additionally, it would require the agent to explore a wider parameter space, increasing training complexity and time costs. Therefore, a reasonable constraint range helps to reduce the parameter search space and make training more efficient.

The prediction horizon $T_{p}$ is the output of the DRL actor network $μ (S_{t} | θ^{μ})$ , which is learned during offline training by maximizing the expected cumulative reward

\begin{matrix} \max_{θ^{μ}} E [\sum_{t} γ^{t - 1} r_{t}] \end{matrix}

(18)

where $S_{t}$ is the state vector and $r_{t}$ is the reward defined in (19). Once trained, the actor network directly maps the current state to an optimal $T_{p}$ within the pre-defined stable range $[0.02, 0.1] s$ . The weighting factor p is a fixed parameter determined offline via trial and error prior to deployment. It is chosen by evaluating tracking performance under control energy constraints (e.g. current limits) to balance tracking accuracy and control effort; $p = 0.01$ is selected as it provides satisfactory performance across the tested operating conditions.

The DRL action space is $T_{p} \in [0.02, 0.1] s$ , which is derived from the stability analysis in Theorem 1. The proof shows that if $T_{p}$ remains within this range, the closed-loop system matrix $W_{Γ}$ is Hurwitz stable. The DRL agent is trained and operated strictly within this safe region.

Reward function

Here, the tracking error $ε (t)$ -related functions are considered as a potential reward function. Then, two subgoals yield a sparse reward to guide the learning agent’s expected error of $ϵ_{1}$ and $ϵ_{2}$ . Furthermore, the thresholds $ϵ_{1}$ and $ϵ_{2}$ are chosen based on specific scenarios. When the output position is close to its reference signal, the positive rewards $β_{1}$ and $β_{2}$ are subsequently given. Additionally, a penalty reward $- β_{3} ε (t)$ is also designed based on the subgoals. Accordingly, the reward function is designed as

\begin{matrix} r = {\begin{matrix} β_{1} - β_{3} | ε (t) |, if 0 \leq | ε (t) | < ϵ_{1} \\ β_{2} - β_{3} | ε (t) |, if ϵ_{1} \leq | ε (t) | \leq ϵ_{2} \\ - β_{3} | ε (t) |, else \end{matrix} \end{matrix}

(19)

Reward coefficients: $β_{1} = 0.5$ , $β_{2} = 0.1$ , $β_{3} = 0.5$ , with thresholds $ϵ_{1} = 0.1 rad$ , $ϵ_{2} = 0.5 rad$ . The reward function directly encourages minimal tracking error, which aligns with the GPC cost function $J_{p}$ that minimizes $\int {(θ_{m} - θ_{r})}^{2} dτ$ . Control effort is implicitly managed through the GPC cost weight p and the constrained action space $T_{p} \in [0.02, 0.1]$ .

Actor and critic network

The function approximators utilized in the reinforcement learning agent are commonly known as the actor and critic networks. Specifically, the critic network comprises three hidden layers with 64, 64, and 32 neurons distributed among the layers. Moreover, the critic network features five units in the input layer, including four for states and one for action. In addition, the actor network consists of two hidden layers with 64 and 32 neurons, respectively, and employs the ReLU function as the activation function for each hidden layer. The structure of the proposed actor-critic network in the DDPG algorithm is shown in Figure 3.

Figure 3.

Schematic of DDPG actor-critic network.

Remark 3. This paper integrates DRL with traditional control strategies with real-time horizon adjustment capabilities to enhance system response and control performance, especially when dealing with dynamic changes in PMSM. In addition, the disturbance observer design further improves robustness and adaptability, especially under load torque changes.

Simulation verification

In this section, MATLAB/Simulink is employed for simulation validation. The parameters of the employed PMSM platform in the simulation and experiment are shown in Table 1. First, the impact of key parameters on the system is analyzed to prepare for reasonable training of DRL. Second, the difference in execution time between the proposed method and the traditional MPC is evaluated. In addition, the P-PI, ADRC, and GPC algorithms are used as control groups to evaluate the dynamic response and disturbance rejection capabilities of the system.

Table 1.

PMSM parameters.

Parameter	Definition	Value
$R_{s}$	Stator resistance	0.36Ω
$L_{d}, L_{q}$	Inductance	0.002 $H$
$n_{p}$	Number of pole pairs	4
J	Moment of inertia	7.0616 $\times 1 0^{- 6} kg \cdot m^{2}$
$Ψ$	Rotor flux	0.0064 Wb
$B_{v}$	Friction coefficient	2.6368 $\times 1 0^{- 6} kg \cdot m^{2} ∕ s$
$n_{N}$	Rated speed	4107 rpm
$I_{N}$	Rated current	7 A
$V_{b}$	Line voltage	24 V

Preparation for DRL training

This section analyzes the impact of two key parameters: the prediction horizon $T_{p}$ and the control input weight coefficient p on system control performance.

As shown in Figure 4(a), as the parameter p increases, the constraints on the control energy become greater, which actually acts as a protection mechanism for the system. The results shown in Figure 4(b) and (c) present the system response curves of two operating conditions, $θ_{r}$ ranges from $0^{\circ}$ to $30 0^{\circ}$ and $θ_{r}$ ranges from $0^{\circ}$ to $150 0^{\circ}$ , under different $T_{p}$ . Obviously, choosing a smaller $T_{p}$ can improve the response speed of the system, but it should be noted that when the value of $T_{p} = 10 ms$ is too small, it may lead to significant overshoot, particularly under large operational conditions. Conversely, choosing a larger $T_{p}$ will cause the system response to become slower under both operating conditions. Therefore, this paper confines the $T_{p}$ within the range of $[0.02, 0.1]$ during agent training to strike a balance.

Figure 4.

Response curves of the GPC with different p and $T_{p}$ : (a) $θ_{r}$ ranges from 0∘ to 500∘, (b) $θ_{r}$ ranges from 0∘ to 300∘, and (c) $θ_{r}$ ranges from 0∘ to 1500∘.

DRL training setup

The DDPG agent is trained offline using a Simulink model of the PMSM servo system. The training environment includes reference steps ranging from $0^{\circ}$ to $150 0^{\circ}$ . Training consists of 50 episodes, each lasting 2 s with a sampling frequency of 10 kHz, resulting in a total of $1 0^{6}$ samples. The training takes approximately 24 hours on an Intel i7 CPU with an RTX 2060 GPU. The state $S_{t} = {θ_{m}, θ_{m_del}, ε, ε_del}$ captures system dynamics without relying on noisy numerical differentiation, as the delayed signals implicitly provide velocity and acceleration information. The reward function r directly penalizes the tracking error $| ε |$ , aligning with the control objective. The thresholds $ϵ_{1}, ϵ_{2}$ and coefficients $β_{1}, β_{2}, β_{3}$ (listed in Table 2) are empirically chosen to encourage convergence to small error regions.

Table 2.

Parameters of different controllers.

Controller	Parameter	Value
P-PI	position loop gain $k_{p}^{p}$	1
	speed loop gains $k_{p}^{v}$ , $k_{i}^{v}$	1.08, 0.03
ADRC	fast tracking factor r	2
	sampling filtering factor h	0.001
	observer bandwidth factor δ	800
	control law gain $α_{1}, α_{2}$	2, 0.1
GPC	horizon $T_{p}$	20 ms, 30 ms
	weighted coefficient p	0.01
	observer bandwidth factor δ	800
GDPC	weighted coefficient p	0.01
	learning rate α	0.001
	discount factor γ	0.95
	replay memory capacity B	1e-5
	minibatch size b	256
	reward function $β_{1}$ , $β_{2}$ , $β_{3}$	0.5, 0.1, 0.5
	subgoals $ϵ_{1}$ , $ϵ_{2}$	0.1, 0.5
	observer bandwidth factor δ	800

Efficient comparison with classical MPC

As shown in Figure 5, within the framework of the position servo system, the proposed method exhibits comparable control performance to MPC, with noticeably different one-step computation times of 8.9 and $50.8 μ s$ , respectively. The proposed method demonstrates a significant computational efficiency advantage over classical MPC. Moreover, it is worth noting that traditional MPC has inherent limitations in terms of model dependence. With the expansion of the prediction horizon, the computational complexity of discrete MPC steadily increases. Therefore, the subsequent discussions primarily compare the validation of control performance with the traditional continuous GPC method.

Figure 5.

Simulation results: (a) position response curves under control period of $100 μ s$ , (b) comparison of calculation efficiency, (c) $θ_{r}$ ranges from 0∘ to 500∘, and (d) the constant load torque of 0.1 Nm.

Simulation verification with different methods

As shown in Figure 5(c), the proposed controller demonstrates dynamic performance comparable to traditional GPC and ADRC. Additionally, it is worth noting that although the P-PI controller is effective, its response speed is slightly slower and the smoothness is reduced compared with the optimal control algorithm. Furthermore, an analysis of Figure 5(d) reveals the excellent capabilities of the proposed controller when confronted with applied load disturbances. It swiftly recovers after the application of external load torque and maintains the overall stability of the system. These results underscore the effectiveness of the proposed control strategy in dealing with challenging real-world conditions.

Remark 4. It is worth noting that while the offline training phase of the DRL agent brings relatively high computational complexity due to the extensive data processing and exploratory learning of control policy optimization, the online operation phase significantly reduces this complexity, allowing the DRL agent to make decisions quickly with minimal computation. In addition, the integration with the Beckhoff controller ensures real-time operation with minimal latency.

Experimental verification

In this section, a series of experiments are conducted to validate the effectiveness and performance enhancement of the proposed control scheme. The experiments involve the combination of a DSP with a Beckhoff controller, enabling rapid deployment and validation of complex control algorithms.

Experimental configuration

The experimental setup, as depicted in Figure 6, comprises several pivotal components: a DSP module equipped with the DRV8305 control driver board, a LAUNCHXL-F28379D module serving as the control core board, a $24 V$ DC power supply to energize the system, and the Beckhoff controller, functioning as the decision-making peripheral interface element. Furthermore, the current loop and position loop are configured with control periods of 20 and $100 μ s$ , respectively. The hardware platform comprises a DSP (LAUNCHXL-F28379D, 200 MHz) for real-time control and a Beckhoff industrial PC (C6930-0070, 1.4 GHz Intel Atom) for DRL inference. Communication is established via EtherCAT with a $100 μ s$ cycle; the DRL inference takes approximately $15 μ s$ , and the total loop delay is less than $5 μ s$ . The load motor operates in torque control mode, generating precise resistance torque $T_{L}$ as per equation (1b). This is an active load system, not merely a commanded signal.

Figure 6.

Experimental platform, showing the Beckhoff IPC, TI DSP, power inverter, PMSM, and load torque generation system.

Experimental verification with different methods

To verify the performance improvement of the proposed method, various experiments are conducted in which the traditional P-PI controller, ADRC (Zuo et al., 2021), GPC¹ ( $T_{p} = 20 ms$ ) (Yang et al., 2015), and GPC² ( $T_{p} = 30 ms$ ) are selected as the control group. The controller parameters for each method are detailed in Table 2. To verify the enhancement of response speed and control precision brought about by the proposed approach, performance validation will be conducted under the following three operating conditions.

Case I— $θ_{r}$ ranges from $0^{\circ}$ to $500^{\circ}$ and $1000^{\circ}$ at the initial moment

As can be seen in Figure 7(a) and (b), it is notable that the proposed approach realizes intelligent horizon adjustment in response to changing operating conditions. The trend of variation $T_{p}$ highlights that while a smaller horizon achieves faster response, it also results in increased overshoot compared to the fixed-horizon GPC strategy. Notably, the proposed controller enhances dynamic responsiveness while avoiding overshoot.

Figure 7.

Experimental results: (a) Case I— $θ_{r}$ ranges from $0 \circ$ to $500 \circ$ , (b) Case I— $θ_{r}$ ranges from $0 \circ$ to $1000 \circ$ , (c) Case II—the constant load torque of $0.1 Nm$ , (d) Case II—the constant load torque of $0.2 Nm,$ (e) Case III—the sine wave load torque of $0.1 \sin (2 πt) Nm$ , and (f) Case IV—the different J.

Case II—apply a constant load of 0.1 Nm at 0.5 s and 0.2 Nm at 1.0 s, respectively

In Figure 7(c) and (d), the results show that when encountering sudden disturbances, the position drop of the PI controller is the largest, followed by the ADRC, while the proposed framework exhibits the lowest position drop and fastest performance recovery. Furthermore, to uphold optimal control precision in the presence of escalating disturbances, the horizon is deliberately minimized. The ESO employed in the proposed composite controller can accurately estimate the lumped disturbance, which guarantees the position tracking accuracy of the servo system when facing complex disturbances.

Case III—apply a sine wave load disturbance with a period of 1 s and an amplitude of 0.1 Nm at 0.5 s

The results described in Figure 7(e) show that when the system is subjected to time-varying disturbance, the PI controller exhibits the highest position oscillation, while the ADRC maintains a certain disturbance rejection capability. The performance of the GPC method is predominantly influenced by the choice of horizon. In contrast, the proposed control approach exhibits a smaller amplitude of position oscillation compared to other controllers. Consequently, the adaptive adjustment of the horizon allows the proposed strategy to exhibit effective disturbance rejection performance under various operating conditions.

Case IV—robustness against parameter perturbations

To verify the efficacy of the proposed method in handling parameter perturbations, a series of experiments are conducted under different J values. The test conditions include a step change in position from $0^{\circ}$ to $50 0^{\circ}$ at $t = 0.1 s$ , a constant load torque of $0.1 Nm$ at $t = 1 s$ , and a time-varying load torque of $0.1 \sin (2 πt) Nm$ at $t = 2 s$ . As shown in Figure 7(f), when the system operates close to the actual J, there is a slight reduction in response speed and disturbance rejection. Conversely, with a large deviation from the actual J, the tracking error increases. Overall, it can be demonstrated that the proposed method effectively maintains control performance even within a certain range of parameter perturbations.

Experiments are conducted with inertia values of $0.5 J_{nom}$ , $0.75 J_{nom}$ , $J_{nom}$ , $1.5 J_{nom}$ , and $2 J_{nom}$ (i.e. from 50% to 200% of the nominal value). Results show that with $J = 2 J_{nom}$ , the settling time increases by approximately 18% compared to the nominal case, while the tracking error remains within 0.1 rad. With $J = 0.5 J_{nom}$ , the response becomes slightly faster but with marginally increased overshoot. The system remains stable throughout the entire tested range. In steady-state conditions with negligible disturbances and accurate model knowledge, well-tuned fixed-horizon GPC or ADRC can achieve comparable performance to GDPC. The advantage of GDPC is most pronounced during dynamic transitions and under simultaneous uncertainties.

To gain more insight quantitatively, the performance indices including the settling time, the position drop, and the position fluctuation are, respectively, presented in Table 3.

Table 3.

Performance indices.

	Settling time		Position drop		Position fluctuation
Method	Case I(a)	Case I(b)	Case II(a)	Case II(b)	Case III
PI-PI	0.19 s	0.22 s	9.87 ∘	14.74 ∘	15.119 ∘
ADRC	0.14 s	0.16 s	3.14 ∘	7.27 ∘	5.13 ∘
GPC ¹	0.12 s	0.13 s	1.31 ∘	4.49 ∘	4.32 ∘
GPC ²	0.15 s	0.16 s	2.03 ∘	5.18 ∘	7.83 ∘
GDPC	0.11 s	0.13 s	1.49 ∘	3.02 ∘	1.71 ∘

Conclusion

This paper has proposed a novel GDPC design scheme for PMSM servo systems that integrates a DRL-based horizon tuning mechanism. In the investigated framework, the horizon has been self-optimized to a certain suitable value according to the operating condition, rather than chosen by human experience. The comparison results have demonstrated that the proposed approach has indeed improved dynamic response and tracking performance. Future work will focus on the multi-objective optimization of the PMSM position servo system in constrained scenarios.

Footnotes

Appendix 1

This subsection provides a rigorous stability analysis of the closed-loop system.

Theorem 1. Considering the PMSM servo system (2) satisfying Assumption 1, if the observer gain is designed as $κ_{i} = δ^{i} {\bar{κ}}_{i}, i = 1, 2, 3$ , where ${\bar{κ}}_{i}$ is chosen such that the characteristic polynomial $λ^{3} + {\bar{κ}}_{1} λ^{2} + {\bar{κ}}_{2} λ + {\bar{κ}}_{3}$ is Hurwitz stable, with the proper selections of δ and $T_{p}$ , the closed-loop system (2)-(9)-(16) is uniformly bounded.

Proof: Defining variables $Λ_{1} = δ^{2} e_{1}$ , $Λ_{2} = δ e_{2}$ , $Λ_{3} = e_{3}$ , $Γ_{1} = ε (t)$ , $Γ_{2} = \overset{\cdot}{ε} (t)$ , the closed-loop system can be specifically depicted as

(20)

\begin{matrix} {\begin{matrix} \overset{\cdot}{Γ} & = W_{Γ} Γ + Q_{Γ} Λ \\ \overset{\cdot}{Λ} & = δ W_{Λ} Λ + D ḋ \end{matrix} \end{matrix}

where $Q_{Γ} = [\begin{matrix} 0 & 0 & 0 \\ 0 & - \frac{k_{2}}{δ^{2}} & - 1 \end{matrix}], Λ = {[Λ_{1}, Λ_{2}, Λ_{3}]}^{⊤},$ $Γ = [\begin{matrix} Γ_{1} \\ Γ_{2} \end{matrix}], W_{Γ} = [\begin{matrix} 0 & 1 \\ - k_{1} & - k_{2} \end{matrix}], W_{Λ} = [\begin{matrix} - {\bar{κ}}_{1} & 1 & 0 \\ - {\bar{κ}}_{2} & 0 & 1 \\ - {\bar{κ}}_{3} & 0 & 0 \end{matrix}] .$

Keeping the selection guideline of the observer parameter ${\bar{κ}}_{i}$ in mind, the matrix $W_{Λ}$ is Hurwitz stable, which means that a matrix $P_{Λ}$ exists such that $W_{Λ}^{⊤} P_{Λ} + P_{Λ} W_{Λ} = - I_{3 \times 3}$ . Furthermore, since the DRL agent guarantees the selection of $T_{p}$ within the range $[0.02, 0.1] s$ , it follows from (16) that $W_{Γ}$ is Hurwitz stable, that is, there exists a matrix $P_{Γ}$ such that $W_{Γ}^{⊤} P_{Γ} + P_{Γ} W_{Γ} = - I_{2 \times 2}$ . For the time-varying case, we consider the worst-case scenario; the Lyapunov function then satisfies $\overset{\cdot}{V} \leq - γ_{o} V$ for all $T_{p}$ in this range, establishing uniform boundedness via ISS arguments.

Construct a candidate Lyapunov function as $V (Γ, Λ) = Γ^{⊤} P_{Γ} Γ + Λ^{⊤} P_{Λ} Λ$ . One can obtain that

(21)

\begin{matrix} λ_{\min} (P_{φ}) Υ \leq V (Γ, Λ) \leq λ_{\max} (P_{φ}) Υ \end{matrix}

where $λ_{\max} (\cdot)$ and $λ_{\min} (\cdot)$ represent the maximum and minimum eigenvalues of the matrix (⋅), $P_{φ} = [\begin{matrix} P_{Γ} & 0 \\ 0 & P_{Λ} \end{matrix}] \in ℝ^{5 \times 5}$ , $Υ = ∥ Γ ∥^{2} + ∥ Λ ∥^{2}$ .

Taking the derivative of the Lyapunov function along (20) yields that

(22)

\begin{matrix} \overset{\cdot}{V} (Γ, Λ) & \leq - \frac{1}{2} ∥ Γ ∥^{2} - (1 - σ) (δ - ν) ∥ Λ ∥^{2} \\ - (δ - ν) σ ∥ Λ ∥ - ∥ 2 P_{Λ} D C ∥ \cdot ∥ Λ ∥ \\ \leq - γ_{o} V (Γ, Λ) - N (Λ) ∥ Λ ∥ \end{matrix}

where $N (Λ) = (δ - ν) σ ∥ Λ ∥ - ∥ 2 P_{Λ} D C ∥, ν = ∥ 2 P_{Γ} Q_{Γ} ∥^{2}, 0 < σ < 1, γ_{o} = \min {\frac{1}{2}, (1 - σ) (δ - ν)} ∕ λ_{\max} (P_{φ})$ .

By defining $μ = ∥ 2 P_{Λ} D C ∥ ∕ [(δ - ν) σ]$ , for $\forall ∥ Λ ∥ \geq μ > 0$ , (22) satisfies

(23)

\begin{matrix} \overset{\cdot}{V} (Γ, Λ) \leq - γ_{o} V (Γ, Λ) \end{matrix}

It can be obtained from Khalil (2002) that when the selection of the observer bandwidth factor satisfies the condition $δ > ν$ , the closed-loop system (20) is uniformly bounded. Furthermore, let $ρ_{1} (∥ (Γ, Λ) ∥) = λ_{\min} (P_{φ}) Υ$ , $ρ_{2} (∥ (Γ, Λ) ∥) = λ_{\max} (P_{φ}) Υ$ . It can be concluded that there exists a time constant $T > 0$ , for $\forall t > T$ , the uniform boundary of the tracking error satisfies the following conditions

(24)

\begin{matrix} ∥ (Γ, Λ) ∥ & \leq ρ_{1}^{- 1} (ρ_{2} (μ)) = μ \sqrt{λ_{\max} (P_{φ}) ∕ λ_{\min} (P_{φ})} \end{matrix}

The proof of Theorem 1 is thus completed.

The above analysis rigorously proves the uniform boundedness of the closed-loop system and identifies the admissible range of $T_{p}$ as $[0.02, 0.1] s$ . This range is critical for ensuring stability when integrating the DRL agent, as it guarantees that the system remains stable under DRL-based tuning.

ORCID iD

Jianliang Mao

Funding

The authors disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported in part by the National Natural Science Foundation of China under grant 62203292.

Declaration of conflicting interests

The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Data availability statement

Data sharing is not applicable to this article as no datasets were generated or analyzed during the current study.

References

Belda

Vosmik

(2016) Explicit generalized predictive control of speed and position of PMSM drives. IEEE Transactions on Industrial Electronics 63(6): 3889–3896.

Cao

Mao

Dong

, et al. (2024) Composite generalized dynamic predictive control with self-tuning horizon for wide-range speed regulation of PMSM drives. IEEE Transactions on Energy Conversion 39(1): 659–674.

Cui

Dong

, et al. (2024) Adaptive horizon seeking for generalized predictive control via deep reinforcement learning with application to DC/DC converters. IEEE Transactions on Circuits and Systems I: Regular Papers 71(5): 2217–2228.

Ding

Hou

Wang

(2023) Disturbance-observer-based second-order sliding mode controller for speed control of PMSM drives. IEEE Transactions on Energy Conversion 38(1): 100–110.

Dong

Mao

Yan

, et al. (2024) Generalized dynamic predictive control for nonlinear systems subject to mismatched disturbances with application to PMSM drives. IEEE Transactions on Industrial Electronics 71(1): 954–964.

Han

(2009) From PID to active disturbance rejection control. IEEE Transactions on Industrial Electronics 56(3): 900–906.

Hou

Zuo

Sun

, et al. (2023) Modified nonlinear active disturbance rejection control for PMSM speed regulation with frequency domain analysis. IEEE Transactions on Power Electronics 38(7): 8126–8134.

Jia

Chen

, et al. (2024) Robust distributed cooperative rendezvous control for heterogeneous marine vehicles using model predictive control. IEEE Transactions on Vehicular Technology 73(8): 11002–11013.

Khalil

(2002) Nonlinear Systems. 3rd ed. Prentice-Hall.

10.

Lee

Yim

(2020) Reinforcement learning-based adaptive PID controller for DPS. Ocean Engineering 216: 107832.

11.

Liu

, et al. (2025) Multiobjective optimization of energy efficiency and fairness in AAV-assisted wireless powered MEC systems: A DRL-based approach. IEEE Internet of Things Journal 12(14): 28758–28775.

12.

Liu

Xue

Zhao

, et al. (2021) Adaptive dynamic programming for control: A survey and recent advances. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51(1): 142–160.

13.

Liu

Tian

Yang

, et al. (2024) Performance-based hierarchical fault-tolerant control for closed-loop systems with multiplicative faults: A data-driven design method. IEEE Transactions on Cybernetics 54(11): 6593–6606.

14.

Liu

(2022) General second-order consensus of discrete-time multiagent systems via Q-learning method. IEEE Transactions on Systems, Man, and Cybernetics: Systems 52(3): 1417–1425.

15.

Liu

Dang

, et al. (2023) Deep reinforcement learning of UAV tracking control under wind disturbances environments. IEEE Transactions on Instrumentation and Measurement 72: 1–13.

16.

Mandra

Galkowski

Rogers

, et al. (2019) Performance-enhanced robust iterative learning control with experimental application to PMSM position tracking. IEEE Transactions on Control Systems Technology 27(4): 1813–1819.

17.

Ouyang

Jin

Xie

, et al. (2025) Adaptive energy management in dual-motor electric vehicles using deep deterministic policy gradient. IEEE Transactions on Transportation Electrification 11(5): 12647–12656.

18.

Pan

Peng

, et al. (2024) Flocking of under-actuated unmanned surface vehicles via deep reinforcement learning and model predictive path integral control. IEEE Transactions on Instrumentation and Measurement 73: 1–11.

19.

Song

Wang

, et al. (2023) Robust model predictive current control for a nine-phase open-end winding PMSM with high computational efficiency. IEEE Transactions on Power Electronics 38(11): 13933–13943.

20.

Tian

Wang

, et al. (2022) Discrete-time repetitive control-based ADRC for current loop disturbances suppression of PMSM drives. IEEE Transactions on Industrial Informatics 18(5): 3138–3149.

21.

Wang

Niu

, et al. (2025) An adaptive sliding mode control for PMSM servo systems beyond the inertia ratio limit. IEEE Transactions on Power Electronics 40(11): 16352–16366.

22.

Chen

Yang

, et al. (2024) Current sensor incipient fault diagnosis in PMSM drive systems using novel interval sliding mode observer. IEEE Transactions on Instrumentation and Measurement 73: 1–11.

23.

Yan

Yang

Sun

, et al. (2018) Robust speed regulation for PMSM servo system with multiple sources of disturbances via an augmented disturbance observer. IEEE/ASME Transactions on Mechatronics 23(2): 769–780.

24.

Yang

Chen

, et al. (2017) Disturbance/uncertainty estimation and attenuation techniques in PMSM drives-a survey. IEEE Transactions on Industrial Electronics 64(4): 3273–3285.

25.

Yang

Zheng

, et al. (2015) Design of a prediction-accuracy-enhanced continuous-time MPC for disturbed systems via a disturbance observer. IEEE Transactions on Industrial Electronics 62(9): 5807–5816.

26.

Yang

Wang

Liu

, et al. (2025) Adaptive linear neuron-augmented ADRC for harmonic current suppression in dual three-phase PMSMs: An optimization strategy. IEEE Journal of Emerging and Selected Topics in Power Electronics 13(5): 6148–6160.

27.

Yue

Zhang

(2023) Robust adaptive integral sliding mode control for two-axis optoelectronic tracking and measuring system based on novel nonlinear extended state observer. IEEE Transactions on Instrumentation and Measurement 72: 1–11.

28.

Zhang

Zhao

, et al. (2025) Adaptive MPC control for in-wheel-motor-drive electric vehicles with semi-active suspension based on binocular vision road identification. IEEE Transactions on Transportation Electrification 11(5): 12136–12148.

29.

Zhang

Zhou

, et al. (2023) A variable self-tuning horizon mechanism for generalized dynamic predictive control on DC/DC boost converters feeding CPLs. IEEE Journal of Emerging and Selected Topics in Power Electronics 11(2): 1650–1660.

30.

Zhang

Yin

, et al. (2021) Adaptive sliding-mode-based speed control in finite control set model predictive torque control for induction motors. IEEE Transactions on Power Electronics 36(7): 8076–8087.

31.

Zheng

Wang

, et al. (2021) Robust continuous model predictive speed and current control for PMSM with adaptive integral sliding-mode approach. IEEE Transactions on Power Electronics 36(12): 14398–14408.

32.

Zuo

Mei

Jiang

, et al. (2021) Linear active disturbance rejection controllers for PMSM speed regulation system considering the speed filter. IEEE Transactions on Power Electronics 36(12): 14579–14592.

A learning-based generalized dynamic predictive control for PMSM position servo system with uncertain dynamics

Abstract

Keywords

Introduction

Problem formulation

PMSM servo model

Control objective

DRL-based GDPC design

Benchmark GPC design

Position prediction model construction

Unknown signal observation

Receding-horizon optimization

Receding-horizon optimization

DRL-based variable horizon of GPC

State space

Action space

Reward function

Actor and critic network

Simulation verification

Preparation for DRL training

DRL training setup

Efficient comparison with classical MPC

Simulation verification with different methods

Experimental verification

Experimental configuration

Experimental verification with different methods

Case I— θ r ranges from 0 ∘ to 500 ∘ and 1000 ∘ at the initial moment

Case II—apply a constant load of 0.1 Nm at 0.5 s and 0.2 Nm at 1.0 s, respectively

Case III—apply a sine wave load disturbance with a period of 1 s and an amplitude of 0.1 Nm at 0.5 s

Case IV—robustness against parameter perturbations

Conclusion

Footnotes

Appendix 1

ORCID iD

Funding

Declaration of conflicting interests

Data availability statement

References

Case I— $θ_{r}$ ranges from $0^{\circ}$ to $500^{\circ}$ and $1000^{\circ}$ at the initial moment