Multi-agent reinforcement learning for traffic signal control: algorithms and robustness analysis

Chunliang Wu, Zhenliang Ma, Inhi Kim

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Reinforcement learning (RL), given its adaptability and generality, has great potential to optimize online traffic signal control strategies. Although studies have proposed various RL-based signal controllers and validated them offline, very few examine the robustness of the trained RL-based controllers when deployed in a dynamic traffic environment. This paper proposed a multi-agent reinforcement learning algorithm for traffic signal control and developed a general multi-agent optimization simulation tool to evaluate different signal control methods. A transfer learning technique is applied to test the robustness of the proposed algorithm and traditional control approaches under different traffic scenarios, including stochastic traffic flow, varying traffic volume, and uncertain sensor data. The experimental results show that the proposed RL-based control method is robust under stochastic traffic flow and variation traffic demand patterns, and it outperforms the fixed-time and vehicle-actuated methods. However, it is unstable in the case of highly noisy sensor data. Also, the trained RL-based controller can continuously learn online and improve its performance by interacting with the dynamic traffic environment, especially when the traffic is congested, and the sensor has noisy observations.

Original languageEnglish
Title of host publication2020 IEEE 23rd International Conference on Intelligent Transportation Systems, ITSC 2020
PublisherIEEE, Institute of Electrical and Electronics Engineers
ISBN (Electronic)9781728141497
DOIs
Publication statusPublished - 2020
EventIEEE Conference on Intelligent Transportation Systems 2020 - Rhodes, Greece
Duration: 20 Sep 202023 Sep 2020
Conference number: 23rd
http://www.itsc2020.org (Website)

Conference

ConferenceIEEE Conference on Intelligent Transportation Systems 2020
Abbreviated titleITSC
CountryGreece
CityRhodes
Period20/09/2023/09/20
Internet address

Cite this