This study presents a multicriteria energy management strategy (EMS) for hybrid energy sources (HES) composed of fuel cell/battery/supercapacitor hybrid power system for logistics trucks, which uses a model-free deep reinforcement learning (DRL) algorithm, namely deterministic strategy gradient (DDPG), to improve the portability and reusability of the system. The proposed EMS is capable of reducing the hydrogen consumption cost, the degradation of fuel cell and battery, as well as sustaining the state of charge (SOC) of battery and supercapacitor. The results of the study found that the total cost was reduced by 9.5% compared to equivalence consumption minimization strategy (ECMS) based EMS under the WLTP driving cycle. A novel deep transfer learning (DTL) based framework for DRL-based EMS is further elaborated and evaluated by four metrics. Two DTL techniques including policy transfer and experience transfer are leveraged to transfer the EMS from original logistic truck to a B-class passenger car powered by fuel cell and battery. The results indicate that the proposed DTL framework is an appropriate approach to transfer EMSs from different vehicle model with various power topology. The convergence speed of DTL-based EMS is apparently accelerated over 50% in comparison to DRL-based EMS. Besides, the fuel optimality, robustness, convergence, and generalization stability for DTL-based EMS is also improved.
EnangWBannisterC. Modelling and control of hybrid electric vehicles (a comprehensive review). Renew Sustain Energ Rev2017; 74: 1210–1239.
2.
LiuJTanJYangW, et al. Better electrochemical performance of PEMFC under a novel pneumatic clamping mechanism. Energy2021; 229: 120796.
3.
TanerT. The novel and innovative design with using H2 fuel of PEM fuel cell: efficiency of thermodynamic analyze. Fuel2021; 302: 121109.
4.
CaiYWuDSunJ, et al. The effect of cathode channel blockages on the enhanced mass transfer and performance of PEMFC. Energy2021; 222: 119951.
5.
OlabiAGWilberforceTAbdelkareemMA. Fuel cell application in the automotive industry and future perspective. Energy2021; 214: 118955.
6.
EttihirKHiguita CanoMBoulonL, et al. Design of an adaptive EMS for fuel cell vehicles. Int J Hydrogen Energy2017; 42: 1481–1489.
7.
TanerT. Energy and exergy analyze of PEM fuel cell: a case study of modeling and simulations. Energy2018; 143: 284–294.
8.
NaqviSAHTanerTOzkaymakM, et al. Hydrogen production through alkaline electrolyzers: a techno-economic and enviro-economic analysis. Chem Eng Technol2023; 46(3): 474–481.
9.
TanerTNaqviSAHOzkaymakM. Techno-economic analysis of a more efficient hydrogen generation system prototype: a case study of PEM electrolyzer with cr-C coated SS304 bipolar plates. Fuel Cells2019; 19(1): 19–26.
10.
HamesYKayaKBaltaciogluE, et al. Analysis of the control strategies for fuel saving in the hydrogen fuel cell vehicles. Int J Hydrogen Energy2018; 43(23): 10810–10821.
11.
OnoriSSerraoLRizzoniG. Hybrid electric vehicles: energy management strategies. Encycl Energy2016; 277(4): 197–213.
12.
TieSFTanCW. A review of energy sources and energy management system in electric vehicles. Renew Sustain Energ Rev2013; 20(4): 82–102.
13.
WangYSunZChenZ. Development of energy management system based on a rule-based power distribution strategy for hybrid power sources. Energy2019; 175: 1055–1066.
14.
XieCOgdenJMQuanS, et al. Optimal power management for fuel cell–battery full hybrid powertrain on a test station. Int J Electr Power Energy Syst2013; 53: 307–320.
15.
WangYSunZChenZ. Rule-based energy management strategy of a lithium-ion battery, supercapacitor and PEM fuel cell system. Energy Proc2019; 158: 2555–2560.
16.
SnoussiJBen ElghaliSBenbouzidM, et al. Auto-adaptive filtering-based energy management strategy for fuel cell hybrid electric vehicles. Energies2018; 11(8): 2118.
17.
HanJKumDParkY. Synthesis of predictive equivalent consumption minimization strategy for hybrid electric vehicles based on closed-form solution of optimal equivalence factor. IEEE Trans Veh Technol2017; 66: 5604–5616.
18.
ZhangFHuXLangariR, et al. Adaptive energy management in automated hybrid electric vehicles with flexible torque request. Energy2021; 214: 118873.
19.
GuoJHeHPengJ, et al. A novel MPC-based adaptive energy management strategy in plug-in hybrid electric vehicles. Energy2019; 175: 378–392.
20.
ZhouWYangLCaiY, et al. Dynamic programming for new energy vehicles based on their work modes. Part II: fuel cell electric vehicles. J Power Sources2018; 407: 92–104.
21.
FletcherTThringRWatkinsonM. An energy management strategy to concurrently optimise fuel consumption & PEM fuel cell lifetime in a hybrid vehicle. Int J Hydrogen Energy2016; 41(46): 21503–21515.
22.
JiangHXuLLiJ, et al. Energy management and component sizing for a fuel cell/battery/supercapacitor hybrid powertrain based on two-dimensional optimization algorithms. Energy2019; 177: 386–396.
23.
TanHZhangHPengJ, et al. Energy management of hybrid electric bus based on deep reinforcement learning in continuous state and action space. Energy Convers Manag2019; 195: 548–560.
24.
WuJHeHPengJ, et al. Continuous reinforcement learning of energy management with deep Q network for a power split hybrid electric bus. Appl Energy2018; 222: 799–811.
25.
XiongRCaoJYuQ. Reinforcement learning-based real-time power management for hybrid energy storage system in the plug-in hybrid electric vehicle. Appl Energy2018; 211: 538–548.
26.
DuGZouYZhangX, et al. Intelligent energy management for hybrid electric tracked vehicles using online reinforcement learning. Appl Energy2019; 251: 113388.
27.
LiLWangXSongJ. Fuel consumption optimization for smart hybrid electric vehicle during a car-following process. Mech Syst Signal Process2017; 87: 17–29.
28.
LeeWJeoungHParkD, et al. A real-time intelligent energy management strategy for hybrid electric vehicles using reinforcement learning. IEEE Access2021; 9: 72759–72768.
29.
LeeHSongCKimN, et al. Comparative analysis of energy management strategies for HEV: dynamic programming and reinforcement learning. IEEE Access2020; 8: 67112–67123.
30.
TangXZhouHWangF, et al. Longevity-conscious energy management strategy of fuel cell hybrid electric vehicle based on deep reinforcement learning. Energy2022; 238: 121593.
31.
WuPPartridgeJAnderliniE, et al. Near-optimal energy management for plug-in hybrid fuel cell and battery propulsion using deep reinforcement learning. Int J Hydrogen Energy2021; 46: 40022–40040.
32.
HuXZouCTangX, et al. Cost-optimal energy management of hybrid electric vehicles using fuel cell/battery health-aware predictive control. IEEE Trans Power Electron2020; 35(1): 382–392.
33.
TanerT. The micro-scale modeling by experimental study in PEM fuel cell. J Therm Eng2017; 3(5): 1515–1526.
34.
Parvez FaraziNZouBAhamedT, et al. Deep reinforcement learning in transportation research: a review. Transp Res Interdiscip Perspect2021; 11: 100425.
35.
GaneshAHXuB. A review of reinforcement learning based energy management systems for electrified powertrains: progress, challenge, and potential solution. Renew Sustain Energ Rev2022; 154: 111833.
36.
TanerT. A flow channel with nafion membrane material design of PEM fuel cell. J Therm Eng2019; 5(5): 456–468.
37.
TanerT. Alternative energy of the future: a technical note of PEM fuel cell water management. J Fundam Renew Energy Appl2015; 5(3): 1–4.
38.
PanSJYangQ. A survey on transfer learning. IEEE Trans Knowl Data Eng2010; 22(10): 1345–1359.
39.
LazaricA. Transfer in reinforcement learning: a framework and a survey. Berlin Heidelberg: Springer, 2012.
40.
TanCSunFKongT, et al. A survey on deep transfer learning. In: KůrkováVManolopoulosYHammerB, et al. (eds) Artificial neural networks and machine learning – ICANN 2018. Cham: Springer, 2018, pp.270–279.
41.
RibaniRMarengoniM. A survey of transfer learning for convolutional neural networks. In: 2019 32nd SIBGRAPI conference on graphics, patterns and images tutorials (SIBGRAPI-T), Rio de Janeiro, Brazil, 28–31 October 2019. New York: IEEE.
42.
LianRTanHPengJ, et al. Cross-type transfer for deep reinforcement learning based hybrid electric vehicle energy management. IEEE Trans Veh Technol2020; 69(8): 8367–8380.
43.
GuoXLiuTTangB, et al. Transfer deep reinforcement learning-enabled energy management strategy for hybrid tracked vehicle. IEEE Access2020; 8: 165837–165848.
44.
ZhouQLiYZhaoD, et al. Transferable representation modelling for real-time energy management of the plug-in hybrid vehicle based on k-fold fuzzy learning and Gaussian process regression. Appl Energy2022; 305: 117853.
45.
AnsareyMShariat PanahiMZiaratiH, et al. Optimal energy management in a dual-storage fuel-cell hybrid vehicle using multi-dimensional dynamic programming. J Power Sources2014; 250: 359–371.
46.
WangJLiuPHicks-GarnerJ, et al. Cycle-life model for graphite-LiFePO4 cells. J Power Sources2011; 196(8): 3942–3948.
