In this paper, a model-free tracking control algorithm is proposed for discrete-time Markov jump singularly perturbed systems. To overcome the severe numerical stiffness and ill-conditioning problems inherent in the singularly perturbed parameter, the original system is first decoupled into fast and slow subsystems via the time-scale separation method. By embedding dynamic reference signals into the subsystems, augmented matrix systems are constructed, thereby converting the original tracking task into a stabilization problem. Subsequently, an improved data-driven policy iteration algorithm is designed. By reconstructing the value function, this algorithm effectively solves the coupled algebraic Riccati equations for the subsystems without relying on any system dynamic information. Furthermore, a suboptimal composite controller is formulated to achieve the desired tracking objective. Finally, the effectiveness and applicability of the proposed control method are validated through simulation on a tunnel diode circuit model.
XuNWuYZongG, et al. Resilient adaptive secure control for MIMO switched CPSs under unknown deception attacks. IEEE Trans Green Commun Netw2026; 10: 1160–1170.
2.
ChuCHeY.A unified neural event-triggered control approach of high-order switched uncertain systems with time-varying state constraints. Robot Intell Autom2026; 46: 290–302. https://doi.org/10.1108/RIA-09-2025-0295
3.
ChenHZongGLiuX, et al. A sub-domain-awareness adaptive probabilistic event-triggered policy for attack-compensated output control of Markov jump CPSs with dynamically matching modes. IEEE Trans Autom Sci Eng2024; 21(3): 4419–4431.
4.
ChenYDongBLiX.Hybrid-triggered control for Markov jump systems with quantizations and hybrid attacks. Proc Inst Mech Eng I J Syst Control Eng2025; 239(2): 232–246.
5.
QiWYangXParkJH, et al. Fuzzy SMC for quantized nonlinear stochastic switching systems with semi-Markovian process and application. IEEE Trans Cybern2022; 52(9): 9316–9325.
6.
WangSZhouJ.Sliding mode control of phase-type TS fuzzy stochastic semi-Markov jump systems. Proc Inst Mech Eng I J Syst Control Eng2024; 238(9): 1657–1667.
7.
QiWZongGSuS-F.Fault detection for semi-Markov switching systems in the presence of positivity constraints. IEEE Trans Cybern2022; 52(12): 13027–13037.
8.
QiWShaMZongG, et al. Adaptive fuzzy control of networked hidden stochastic switching power systems under cyber attacks. IEEE Trans Cybern2025; 55(7): 3182–3192.
9.
ShenFJuPShahidehpourM, et al. Singular perturbation for the dynamic modeling of integrated energy systems. IEEE Trans Power Syst2020; 35(3): 1718–1728.
10.
LiXMaoWLiJ, et al. Forced event-triggered finite-time extended dissipative control for a class of singularly perturbed switched systems. ISA Trans2025; 162: 209–216.
11.
ShenHLiuYAWangJ, et al. Sliding-mode control for IT2 fuzzy nonlinear singularly perturbed systems and its application to electric circuits: a dynamic event-triggered mechanism. IEEE Trans Syst Man Cybern Syst2023; 53(7): 4077–4090.
12.
WangYShiPYanH.Reliable control of fuzzy singularly perturbed systems and its application to electronic circuits. IEEE Trans Circuits Syst I Regul Pap2018; 65(10): 3519–3528.
13.
ZhaoJYangCGaoW, et al. Adaptive optimal output regulation of interconnected singularly perturbed systems with application to power systems. IEEE/CAA J Automat Sin2024; 11(3): 595–607.
14.
ShenCXuJChengJ, et al. Protocol-based SMC for singularly perturbed systems with switching parameters and deception attacks. Inf Sci2024; 679: 121089.
15.
HuYCaiCLinH, et al. Fuzzy-model-based control for networked singularly perturbed systems: the asynchronous weighted try-once-discard protocol case. IEEE Trans Fuzzy Syst2024; 32(5): 2713–2724.
16.
WangJZhangLZhaoX, et al. Improved results on reachable set estimation for singularly perturbed systems with time-varying delay. J Franklin Inst2024; 361(1): 125–134.
17.
SunTLiangDSongY.Singular-perturbation-based nonlinear hybrid control of redundant parallel robot. IEEE Trans Ind Electron2018; 65(4): 3326–3336.
18.
ShenHLiFXuS, et al. Slow state variables feedback stabilization for semi-Markov jump systems with singular perturbations. IEEE Trans Automat Contr2018; 63: 2709–2714.
19.
SuWCGajicZShenXM.The exact slow-fast decomposition of the algebraic Ricatti equation of singularly perturbed systems. IEEE Trans Automat Contr1992; 37(9): 1456–1459.
20.
KecmanVBingulacSGajicZ.Eigenvector approach for order-reduction of singularly perturbed linear-quadratic optimal control problems. Automatica1999; 35(1): 151–158.
21.
ChowJKokotovicP.A decomposition of near-optimum regulators for systems with slow and fast modes. IEEE Trans Automat Contr1976; 21(5): 701–705.
22.
XuYWangXWangL, et al. Learning control for flexible manipulators with varying loads: a composite method with robust adaptive dynamic programming and robust sliding mode control. Electronics2022; 11: 956.
23.
LiuDWeiQ.Policy iteration adaptive dynamic programming algorithm for discrete-time nonlinear systems. IEEE Trans Neural Netw Learn Syst2014; 25(3): 621–634.
ShenHWuJWangJ, et al. Adversarial dynamic games for Markov jump systems: a policy iteration Q-learning method. Automatica2026; 183: 112591.
26.
ShenHWangYWuJ, et al. Secure control for Markov jump cyber-physical systems subject to malicious attacks: a resilient hybrid learning scheme. IEEE Trans Cybern2024; 54(11): 7068–7079.
27.
XuNTangLAl-BarakatiAA.Fixed-time optimal bipartite containment fault-tolerant control for multi-agent systems under multiple faults and saturated actuation. Math Comput Simul2026; 245: 242–259.
28.
KiumarsiBLewisFLJiangZP. control of linear discrete-time systems: off-policy reinforcement learning. Automatica2017; 78: 144–152.
29.
WangYAhnCKYanH, et al. Fuzzy control and filtering for nonlinear singularly perturbed Markov jump systems. IEEE Trans Cybern2021; 51(1): 297–308.
30.
AhmadiPRahmaniMShahmansoorianA.Model-free inverse optimal control for completely unknown nonlinear systems by adaptive dynamic programming. IEEE Trans Syst Man Cybern Syst2025; 55(7): 4764–4775.
31.
ZongGXieHYangD, et al. Adaptive fuzzy tracking control for switched nonlinear systems under FDI attacks and input saturation: A flexible transient performance approach. IEEE Trans Cybern2024; 54(12): 7479–7488.
32.
WangYZongGZhaoX, et al. Adaptive practical fixed-time synchronized tracking control of ASV with prescribed performance. Automatica2024; 166: 111716.
33.
YangDZongGShiY, et al. Adaptive tracking control of hybrid switching Markovian systems with its applications. SIAM J Control Optim2023; 61(2): 434–457.
34.
PengX-JHeYLiuZ, et al. Time-varying formation H∞ tracking control and optimization for delayed multi-agent systems with exogenous disturbances. IEEE Trans Autom Sci Eng2025; 22: 5637–5647.
35.
WangZXiangXDuanY, et al. Adversarial deep reinforcement learning based robust depth tracking control for underactuated autonomous underwater vehicle. Eng Appl Artif Intell2024; 130: 107728.
36.
WangJPengCParkJH, et al. Reinforcement learning-based near optimization for continuous-time Markov jump singularly perturbed systems. IEEE Trans Circuits Syst II Express Briefs2023; 70(6): 2026–2030.
37.
ShiXLiYDuC, et al. Reinforcement learning-based optimal control for Markov jump systems with completely unknown dynamics. Automatica2025; 171: 111886.
38.
VijayakumarMSakthivelRMohammadzadehA, et al. Proportional integral observer based tracking control design for Markov jump systems. Appl Math Comput2021; 410: 126467.
39.
YuanSShiT.Adaptive event-triggered output tracking control for Markovian jump systems: a robust H2/H∞ MPC approachJ Franklin Inst2023; 360(17): 12808–12828.
40.
ZhengDDXieWFChaiT, et al. Identification and trajectory tracking control of nonlinear singularly perturbed systems. IEEE Trans Ind Electron2017; 64(5): 3737–3747.
41.
ZhangKSuRZhangH.A novel resilient control scheme for a class of Markovian jump systems with partially unknown information. IEEE Trans Cybern2022; 52(8): 8191–8200.
42.
WangJWuJCaoJ, et al. Nonfragile output feedback tracking control for Markov jump fuzzy systems based on integral reinforcement learning scheme. IEEE Trans Cybern2023; 53(7): 4521–4530.
43.
NaiduDSRaoAK.Singular perturbation analysis of discrete control systems. Springer, 1985.
44.
CostaOLVFragosoMDMarquesRP.Discrete-time Markov jump linear systems. Springer, 2005.
45.
LiWJiaGWangY, et al. Reinforcement learning-based composite suboptimal control for Markov jump singularly perturbed systems with unknown dynamics. Math Methods Appl Sci2024; 47(14): 11551–11564.
46.
XuJChengJYanH, et al. Dynamic event-triggered control for semi-Markov singularly perturbed systems with generally transition rates. IEEE Trans Syst Man Cybern Syst2024; 54(1): 225–235.
