Research on relieve of traffic congestion based on optimization of DQN algorithm in CamTra recognition system

Abstract

This research presents a method of calculation for traffic signal plans based on the DQN (Deep Q-Leaning Network) algorithm to relieve traffic congestion in the city during rush hour. It is a very important part of our independently developed Camera Tracking System (CamTra), which has been applied in various fields such as traffic big data processing, drowning warning, factory intelligence optimization, and precise logistics simulation. With the increased number of cars, the occurrence of traffic congestion is rapidly increasing. The most important difficulty addressed in this paper is the analysis of the signal plan for tempo (speed and spacing) and green wave transmission. In addition to this, the research further optimized other accompanying factors such as action space and network structure, to achieve the best adaptability for signals. With the increase of adaptability for signal, the effectiveness of the ε-greedy strategy has also been further enhanced. The improved algorithm associates the ε-greedy strategy with the cumulative reward sequence, which realizes the adaptive control of the ε-greedy strategy and balances the relationship between exploration and performance. Finally, with the experiment of SUMO (Simulation of Urban Mobility), the performance of the enhanced DQN algorithm in single- and double-crossroads traffic signal control was verified. In comparison with the traditional static signal strategy, the enhanced DQN algorithm performed better in relieving traffic congestion.

Keywords

Relieve traffic congestion green wave DQN algorithm ε-greedy strategy experiment of SUMO

Get full access to this article

View all access options for this article.

References

Kaffash

Nguyen

Zhu

. Big data algorithms and applications in intelligent transportation system: a review and bibliometric analysis. Int J Prod Econ 2021; 231: 107868.

Lin

Wang

Intelligent transportation system (ITS): concept, challenge and opportunity. In: 2017 IEEE 3rd international conference on big data security on cloud (bigdatasecurity), IEEE international conference on high performance and smart computing (HPSC), and IEEE international conference on intelligent data and security (IDS), Beijing, China, 26–28 May 2017, pp.167–172. New York: IEEE.

Muthuramalingam

Bharathi

Rakesh Kumar

, et al. IoT based intelligent transportation system (IoT-ITS) for global perspective: a case study. In: Balas

Solanki

Kumar

, et al. (eds) Internet of things and big data analytics for smart generation. Cham: Springer, 2019, pp.279–300.

Lee

Shin

. A survey of intelligent transportation systems. In: 2011 third international conference on computational intelligence, communication systems and networks, Bali, Indonesia, 26–28 July 2011, pp.332–337. New York: IEEE.

Qian

Zeng

Wang

, et al. A traffic flow model considering influence of car-following and its echo characteristics. Nonlinear Dyn 2017; 28: 178–192.

Alemzadeh

Moslemi

Sharma

, et al. Adaptive traffic control with deep reinforcement learning: towards state-of-the-art and beyond. arxiv preprint arxiv: 2007.10960, 2020.

Ducrocq

Farhi

Deep reinforcement Q-learning for intelligent traffic signal control with partial detection. Int J Intell Transp Syst Res 2023; 21(1): 192–206.

Zeng

Qian

Yin

, et al. A multi value cellular automata model for multi lane traffic flow under lagrange coordinate. Comput Math Organ Theory 2021; 28: 178–192.

Shabestary

Abdulhai

Adaptive traffic signal control with deep reinforcement learning and high dimensional sensory inputs: case study and comprehensive sensitivity analyses. IEEE Trans Intell Transp Syst 2022; 23(11): 21–35.

10.

Touhbi

Babram

Nguyen-Huu

, et al. Adaptive traffic signal control: exploring reward definition for reinforcement learning. Procedia Comput Sci 2017; 109: 513–520.

11.

Zeng

Qian

, et al. Congestion and energy consumption of heterogeneous traffic flow mixed with intelligent connected vehicles and platoons. Physica A 2023; 609: 128331.

12.

Zhang

Qian

Zeng

, et al. Hybrid characteristics of heterogeneous traffic flow mixed with electric vehicles considering the amplitude of acceleration and deceleration. Physica A 2023; 614: 128556.

13.

Xiao

Liu

, et al. Leveraging deep reinforcement learning for traffic engineering: a survey. IEEE Commun Surv Tutor 2021; 23(4): 2064–2097.

14.

Jiang

Liu

A reinforcement double deep Q-network with prioritised experience replay for rolling bearing fault diagnosis. Meas Sci Technol 2023; 34(12): 125–133.

15.

Roy

Borkar

Karandikar

, et al. Online reinforcement learning of optimal threshold policies for Markov decision processes. IEEE Trans Automat Contr 2021; 67(7): 3722–3729.

16.

Wei

Jahromi

Luo

, et al. Model-free reinforcement learning in infinite-horizon average-reward Markov decision processes. Proc Mach Learn Res 2020; 119: 10170–10180.

17.

Clifton

Laber

Q-learning: theory and applications. Annu Rev Stat Appl 2020; 7: 279–301.

18.

Fan

Wang

Xie

, et al. A theoretical analysis of deep Q-learning. Proc Mach Learn Res 2020; 120: 486–489.

19.

Jang

Kim

Harerimana

, et al. Q-learning algorithms: a comprehensive classification and applications. IEEE Access 2019; 7: 133653–133667.

20.

, et al. Cognitive electronic jamming decision-making method based on improved Q-learning algorithm. Int J Aerosp Eng 2021; 2021: 1–2.

21.

Zhang

Wang

, et al. On the convergence and sample complexity analysis of deep Q-networks with ε-greedy exploration. Adv Neural Inf Process Syst 2024; 36: 4.

22.

Zhang

Tian

Zhang

, et al. Neighborhood cooperative multiagent reinforcement learning for adaptive traffic signal control in epidemic regions. IEEE Trans Intell Transp Syst 2022; 23(12): 25157–25168.

23.

Chen

Wang

, et al. A two-stage decomposition-reinforcement learning optimal combined short-time traffic flow prediction model considering multiple factors. Appl Sci 2022; 12(16): 7978.

24.

Chen

, et al. Research on optimized GA-SVM vehicle speed prediction model based on driver-vehicle-road-traffic system. Sci China Technol Sci 2018; 61: 782–790.