We consider the maintenance process of gas turbines used in the Oil and Gas industry: the capital parts are first removed from the gas turbines and replaced by parts of the same type taken from the warehouse; then, they are repaired at the workshop and returned to the warehouse for use in future maintenance events. Experience-based rules are used to manage the flow of the parts for a profitable gas turbine operation. In this article, we formalize the part flow management as a sequential decision problem and propose reinforcement learning for its solution. An application to a scaled-down case study derived from real industrial practice shows that reinforcement learning can find policies outperforming those based on experience-based rules.
YangKHeCHuangQet al. Very high cycle fatigue behaviors of a turbine engine blade alloy at various stress ratios. Int J Fatigue2017; 99: 35–43.
2.
PetersJRitchieR.Influence of foreign-object damage on crack initiation and early crack growth during high-cycle fatigue of Ti-6Al-4V. Eng Fract Mech2000; 67(3): 193–207.
3.
BoyceBRitchieR.Effect of load ratio and maximum stress intensity on the fatigue threshold in Ti-6Al-4V. Eng Fract Mech2001; 68(2): 129–147.
4.
TarabrinAPSchurovskyVABodrovAet al. An analysis of axial compressor fouling and a blade cleaning method. J Turbomach1998; 120(2): 256–261.
5.
KurzRBrunK.Fouling mechanisms in axial compressors. J Eng Gas Turbine Power2012; 134(3): 032401.
6.
MoriniMPinelliMSpinaPet al. Influence of blade deterioration on compressor and turbine performance. J Eng Gas Turbine Power2010; 132(3): 032401.
7.
EliazNShemeshGLatanisionR.Hot corrosion in gas turbine components. Eng Fail Anal2002; 9(1): 31–43.
8.
GowardG.Progress in coatings for gas turbine airfoils. Surf Coat Technol1998; 108: 73–79.
9.
CompareMMartiniFMattafirriSet al. Semi-Markov model for the oxidation degradation mechanism in gas turbine nozzles. IEEE Trans Reliab2016; 65(2): 574–581.
10.
BalevicDHartmanSYoumansR.Heavy-duty gas turbine operating and maintenance considerations. Atlanta, GA: GE Energy, 2010.
11.
NgIParryGWildPet al. Complex engineering service systems: concepts and research. London: Springer, 2011.
12.
WangW.A model for maintenance service contract design, negotiation and optimization. Eur J Oper Res2010; 201(1): 239–246.
13.
MurthyDAsgharizadehE.Optimal decision making in a maintenance service operation. Eur J Oper Res1999; 116(2): 259–273.
14.
BollapragadaSGuptaALawsiriratC.Managing a portfolio of long term service agreements. Eur J Oper Res2007; 182(3): 1399–1411.
15.
HuQBoylanJEChenHet al. OR in spare parts management: a review. Eur J Oper Res2018; 266(2): 395–414.
16.
SuttonRSBartoAG.Introduction to reinforcement learning, vol. 135. Cambridge: MIT Press, 1998.
17.
LieckensKTColenPJLambrechtMR.Network and contract optimization for maintenance services with remanufacturing. Comput Oper Res2015; 54: 232–244.
18.
KurzJ.Capacity planning for a maintenance service provider with advanced information. EurJ Oper Res2016; 251(2): 466–477.
19.
JinTTianZXieM.A game-theoretical approach for optimizing maintenance, spares and service capacity in performance contracting. Int J Prod Econ2015; 161: 31–43.
20.
KeizerMCOTeunterRHVeldmanJ.Joint condition-based maintenance and inventory optimization for systems with multiple components. Eur J Oper Res2017; 257(1): 209–222.
21.
Van HorenbeekABuréJCattrysseDet al. Joint maintenance and inventory optimization systems: a review. Int J Prod Econ2013; 143(2): 499–508.
22.
JaturonnateeJMurthyDBoondiskulchokR.Optimal preventive maintenance of leased equipment with corrective minimal repairs. Eur J Oper Res2006; 174(1): 201–215.
23.
CompareMBellaniLCobelliEet al. Reinforcement learning-based flow management of gas turbine parts under stochastic failures. Int J Adv Manuf Technol2018; 99(9–12): 2981–2992.
24.
KaelblingLPLittmanMLMooreAW.Reinforcement learning: a survey. J Artif Intell Res1996; 4: 237–285.
KuznetsovaELiYFRuizCet al. Reinforcement learning for microgrid energy management. Energy2013; 59: 133–146.
27.
RahimiyanMMashhadiHR.An adaptive Q-learning algorithm developed for agent-based computational modeling of electricity market. IEEE Trans Syst Man Cybern C Appl Rev2010; 40(5): 547.
28.
BardeSRYacoutSShinH.Optimal preventive maintenance policy based on reinforcement learning of a fleet of military trucks. J Intell Manuf2019; 30(1): 147–161.
29.
AissaniNBeldjilaliBTrentesauxD.Dynamic scheduling of maintenance tasks in the petroleum industry: a reinforcement approach. Eng Appl Artif Intell2009; 22(7): 1089–1103.
30.
PontrandolfoPGosaviAOkogbaaOGet al. Global supply chain management: a reinforcement learning approach. Int J Prod Res2002; 40(6): 1299–1317.
31.
GiannoccaroIPontrandolfoP.Inventory management in supply chains: a reinforcement learning approach. Int J Prod Econ2002; 78(2): 153–161.
32.
KimCJunJBaekJet al. Adaptive inventory control models for supply chain management. Int J Adv Manuf Technol2005; 26(9–10): 1184–1192.
33.
LiuQDongMLvWet al. Manufacturing system maintenance based on dynamic programming model with prognostics information. J Intell Manuf2019; 30(3): 1155–1173.
34.
LiZDingZWangM.Optimal bidding and operation of a power plant with solvent-based carbon capture under a CO2 allowance market: a solution with a reinforcement learning-based Sarsa temporal-difference algorithm. Engineering2017; 3(2): 257–265.
35.
WhiteheadSDLinLJ.Reinforcement learning of non-Markov decision processes. Artif Intell1995; 73(1–2): 271–306.
36.
WangYHLiTHSLinCJ.Backward Q-learning: the combination of Sarsa algorithm and Q-learning. Eng Appl Artif Intell2013; 26(9): 2184–2193.
37.
GuoMLiuYMalecJ.A new Q-learning algorithm based on the metropolis criterion. IEEE Trans Syst Man Cybern B2004; 34(5): 2140–2143.
38.
HwangKSTanSWChenCC.Cooperative strategy based on adaptive Q-learning for robot soccer systems. IEEE Trans on Fuzzy Syst2004; 12(4): 569–576.
