Solving space differential game in an unknown environment remains a challenging problem. This article proposes a pretrained fuzzy actor–critic learning algorithm for dealing with the space pursuit-evasion game in fixed time. It is supposed that the research objects are two agents including one pursuer and one evader in space. A virtual environment, which is defined as the known part of the real environment, is utilized for deriving optimal strategies of the pursuer and the evader, respectively. Through employing the fuzzy inference system, a pretrained process, which is based on the genetic algorithm, is designed to obtain the initial consequent set of the pursuer and the evader. Besides, an actor–critic framework is applied to finely learn the suitable consequent set of the pursuer and evader in the real environment. Numerical experimental results validate the effectiveness of the proposed algorithms on improving the ability of the agents to adapt to the real environment.
NashJ. Non-cooperative games. Ann Mathematics■■■; 54(2): 286–295.
2.
CruzJBChenCI. Series nash solution of two-person, nonzero-sum, linear-quadratic differential games. J Optimization Theor Appl1971; 7(4): 240–257.
3.
HoYCStarrAW. Further properties of nonzero-sum differential games. J Optimization Theor Appl1969; 3(4): 207–219.
4.
GuelmanM. Proportional navigation with a maneuvering target. IEEE Trans Aerospace Electron Syst1972; 8(3): 364–371.
5.
BeckerK. Closed-form solution of pure proportional navigation. IEEE Trans Aerospace Electron Syst1990; 26(3):526–533.
6.
BerkovitzLD. The existence of value and saddle point in games of fixed duration. SIAM J Control Optimization1985; 23(2): 172–196.
7.
BreitnerMHPeschHJGrimmW. Complex differential games of pursuit-evasion type with state constraints, part 2: numerical computation of optimal open-loop strategies. J Optimization Theor Appl1993; 78(3):443–463.
8.
HorieKConwayBA. Optimal fighter pursuit-evasion maneuvers found via two-sided optimization. J Guidance, Control Dyn2006; 29(1): 105–112.
9.
PontaniMConwayBA. Numerical solution of the three-dimensional orbital pursuit-evasion game. J Guidance Control Dyn2009; 32(32): 474–487.
10.
PontaniMConwayBStupikJ. Optimal pursuit/evasion spacecraft trajectories in the hill reference frame. In: Aas/aiaa Astrodynamics Specialist Conference, Minneapolis, Minnesota, 13–16 August, 2013.
11.
HorieKConwayB. Genetic algorithm preprocessing for numerical solution of differential games problems. J Guidance Control Dyn2004; 27: 1075–1078.
12.
JagatASinclairAJ. Optimization of spacecraft pursuit-evasion game trajectories in the euler-hill reference frame. In: Aiaa/aas Astrodynamics Specialist Conference,San Diego, CA, 4–7August 2014.
13.
JagatASinclairAJ. Nonlinear control for spacecraft pursuit-evasion game using state-dependent riccati equation method. IEEE Trans Aerospace Electron Syst2017; 53(99): 1.
14.
ShenDPhamKBlaschE, et al.Pursuit-evasion orbital game for satellite interception and collision avoidance. Proc SPIE - Int Soc Opt Eng2011; 8044(7): 284–287.
15.
ZhangQChenJSunS, et al.The differential game barrier model for spacecraft under target maneuvering based on minimum error. In: SPIE Proc,Orlando, Florida, 3 May 2007.
16.
SunSZhangQYiC. Numerical solution for a class of pursuit-evasion problem in low earth orbit. In 2013 9th Asian Control Conf (Ascc), Istanbul, 1–6 June 2013.
17.
BartoAG. Reinforcement learning. A Bradford Book1998; 15(7):665–685.
18.
SuttonRSBartoAGWilliamsRJ. Reinforcement learning is direct adaptive optimal control. IEEE Control Syst Mag1992; 12(2): 19–22.
Van HasseltHGuezASilverD. Deep reinforcement learning with double q-learning. In Thirtieth AAAI Conference on Artificial Intelligence, Phoenix, AZ,12–17 February, 2016.
21.
MengJErDengC. Obstacle avoidance of a mobile robot using hybrid learning approach. IEEE Trans Ind Electronics2005; 52(3): 898–905.
22.
DaiXLiCKRadAB. An approach to tune fuzzy controllers based on reinforcement learning for autonomous vehicle control. IEEE Trans Intell Transportation Syst2005; 6(3): 285–293.
23.
XiaoHLiao LiZhouF. Mobile robot path planning based on q-ann. 2007 IEEE International Conference on Automation and Logistics. Jinan: China,18–21, August, 2007, pp. 2650–2654.
24.
HungSGivigiS. A q-learning approach to flocking with uavs in a stochastic environment. IEEE Trans Cybernetics2016; 47: 1–12.
IzzoDMrtensMPanB. A survey on artificial intelligence trends in spacecraft guidance dynamics and control. Astrodynamics2019; 4: 287–299.
28.
BilginATKadioglu-UrtisE. An approach to multi-agent pursuit evasion games using reinforcement learning. In 2015 International Conference on Advanced Robotics (ICAR),Istanbul, Turkey, 27–31 July 2015, pp. 164–169.
29.
WangXShiPWenC, et al.An algorithm of reinforcement learning for maneuvering parameter self-tuning applying in satellite cluster. Math Probl Eng2020; 2020: 1–17.
30.
WangXShiPZhaoY, et al.A pre-trained fuzzy reinforcement learning method for the pursuing satellite in a one-to-one game in space. Sensors (Switzerland)2020; 20(8): 2253.
31.
WangXShiPWenC, et al.An algorithm of reinforcement learning for maneuvering parameter self-tuning applying in satellite cluster. Math Probl Eng2020; 2020(5): 1–17.
32.
AnalikwuCVSchwartzHM. Reinforcement learning in the guarding a territory game. In 2016 IEEE International Conference on Fuzzy Systems,Vancouver, Canada, 24–29 July, 2016, pp. 1007–1014.
33.
AwhedaMDSchwartzHM. A residual gradient fuzzy reinforcement learning algorithm for differential games. Int J Fuzzy Syst2017; 19(4): 1058–1076.
