MARL for decentralized electric vehicle charging coordination with V2V energy exchange

Jiarong Fan; Hao Wang; Ariel Liebman

doi:10.1109/IECON51785.2023.10312315

MARL for decentralized electric vehicle charging coordination with V2V energy exchange

Jiarong Fan, Hao Wang, Ariel Liebman

Department of Data Science & AI

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

2 Citations (Scopus)

Abstract

Effective energy management of electric vehicle (EV) charging stations is critical to supporting the transport sector's sustainable energy transition. This paper addresses the EV charging coordination by considering vehicle- to- vehicle (V2V) energy exchange as the flexibility to harness in EV charging stations. Moreover, this paper takes into account EV user experiences, such as charging satisfaction and fairness. We propose a Multi-Agent Reinforcement Learning (MARL) approach to coordinate EV charging with V2V energy exchange while considering uncertainties in the EV arrival time, energy price, and solar energy generation. The exploration capability of MARL is enhanced by introducing parameter noise into MARL's neural network models. Experimental results demonstrate the superior performance and scalability of our proposed method compared to traditional optimization baselines. The decentralized execution of the algorithm enables it to effectively deal with partial system faults in the charging station.

Original language	English
Title of host publication	IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society
Editors	Xing Zhu, Antonio Luque, Weihai Chen
Place of Publication	Piscataway NJ USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Number of pages	6
ISBN (Electronic)	9798350331820
ISBN (Print)	9798350331837
DOIs	https://doi.org/10.1109/IECON51785.2023.10312315
Publication status	Published - 2023
Event	Annual Conference of the IEEE Industrial Electronics Society 2023 - Singapore, Singapore Duration: 16 Oct 2023 → 19 Oct 2023 Conference number: 49th https://ieeexplore.ieee.org/xpl/conhome/10311571/proceeding (Proceedings) https://www.iecon2023.org/ (Website)

Publication series

Name	IECON Proceedings (Industrial Electronics Conference)
Publisher	IEEE, Institute of Electrical and Electronics Engineers
ISSN (Print)	2162-4704
ISSN (Electronic)	2577-1647

Conference

Conference	Annual Conference of the IEEE Industrial Electronics Society 2023
Abbreviated title	IECON 2023
Country/Territory	Singapore
City	Singapore
Period	16/10/23 → 19/10/23
Internet address	https://ieeexplore.ieee.org/xpl/conhome/10311571/proceeding (Proceedings) https://www.iecon2023.org/ (Website)

Keywords

Electric vehicle
energy management
multi-agent reinforcement learning (MARL)
vehicle-to-grid
vehicle-to-vehicle

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1109/IECON51785.2023.10312315

Cite this

Fan, J., Wang, H., & Liebman, A. (2023). MARL for decentralized electric vehicle charging coordination with V2V energy exchange. In X. Zhu, A. Luque, & W. Chen (Eds.), IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society (IECON Proceedings (Industrial Electronics Conference)). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/IECON51785.2023.10312315

Fan, Jiarong ; Wang, Hao ; Liebman, Ariel. / MARL for decentralized electric vehicle charging coordination with V2V energy exchange. IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society. editor / Xing Zhu ; Antonio Luque ; Weihai Chen. Piscataway NJ USA : IEEE, Institute of Electrical and Electronics Engineers, 2023. (IECON Proceedings (Industrial Electronics Conference)).

@inproceedings{7aabe11707ab4f56898db90ac62f6281,

title = "MARL for decentralized electric vehicle charging coordination with V2V energy exchange",

abstract = "Effective energy management of electric vehicle (EV) charging stations is critical to supporting the transport sector's sustainable energy transition. This paper addresses the EV charging coordination by considering vehicle- to- vehicle (V2V) energy exchange as the flexibility to harness in EV charging stations. Moreover, this paper takes into account EV user experiences, such as charging satisfaction and fairness. We propose a Multi-Agent Reinforcement Learning (MARL) approach to coordinate EV charging with V2V energy exchange while considering uncertainties in the EV arrival time, energy price, and solar energy generation. The exploration capability of MARL is enhanced by introducing parameter noise into MARL's neural network models. Experimental results demonstrate the superior performance and scalability of our proposed method compared to traditional optimization baselines. The decentralized execution of the algorithm enables it to effectively deal with partial system faults in the charging station.",

keywords = "Electric vehicle, energy management, multi-agent reinforcement learning (MARL), vehicle-to-grid, vehicle-to-vehicle",

author = "Jiarong Fan and Hao Wang and Ariel Liebman",

note = "Funding Information: This work was supported in part by the Australian Research Council (ARC) Discovery Early Career Researcher Award (DECRA) under Grant DE230100046. *Corresponding author: Hao Wang. Publisher Copyright: {\textcopyright} 2023 IEEE.; Annual Conference of the IEEE Industrial Electronics Society 2023, IECON 2023 ; Conference date: 16-10-2023 Through 19-10-2023",

year = "2023",

doi = "10.1109/IECON51785.2023.10312315",

language = "English",

isbn = "9798350331837",

series = "IECON Proceedings (Industrial Electronics Conference)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

editor = "Xing Zhu and Antonio Luque and Weihai Chen",

booktitle = "IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society",

address = "United States of America",

url = "https://ieeexplore.ieee.org/xpl/conhome/10311571/proceeding, https://www.iecon2023.org/",

}

Fan, J, Wang, H & Liebman, A 2023, MARL for decentralized electric vehicle charging coordination with V2V energy exchange. in X Zhu, A Luque & W Chen (eds), IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society. IECON Proceedings (Industrial Electronics Conference), IEEE, Institute of Electrical and Electronics Engineers, Piscataway NJ USA, Annual Conference of the IEEE Industrial Electronics Society 2023, Singapore, Singapore, 16/10/23. https://doi.org/10.1109/IECON51785.2023.10312315

MARL for decentralized electric vehicle charging coordination with V2V energy exchange. / Fan, Jiarong; Wang, Hao; Liebman, Ariel.
IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society. ed. / Xing Zhu; Antonio Luque; Weihai Chen. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers, 2023. (IECON Proceedings (Industrial Electronics Conference)).

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - MARL for decentralized electric vehicle charging coordination with V2V energy exchange

AU - Fan, Jiarong

AU - Wang, Hao

AU - Liebman, Ariel

N1 - Conference code: 49th

PY - 2023

Y1 - 2023

N2 - Effective energy management of electric vehicle (EV) charging stations is critical to supporting the transport sector's sustainable energy transition. This paper addresses the EV charging coordination by considering vehicle- to- vehicle (V2V) energy exchange as the flexibility to harness in EV charging stations. Moreover, this paper takes into account EV user experiences, such as charging satisfaction and fairness. We propose a Multi-Agent Reinforcement Learning (MARL) approach to coordinate EV charging with V2V energy exchange while considering uncertainties in the EV arrival time, energy price, and solar energy generation. The exploration capability of MARL is enhanced by introducing parameter noise into MARL's neural network models. Experimental results demonstrate the superior performance and scalability of our proposed method compared to traditional optimization baselines. The decentralized execution of the algorithm enables it to effectively deal with partial system faults in the charging station.

AB - Effective energy management of electric vehicle (EV) charging stations is critical to supporting the transport sector's sustainable energy transition. This paper addresses the EV charging coordination by considering vehicle- to- vehicle (V2V) energy exchange as the flexibility to harness in EV charging stations. Moreover, this paper takes into account EV user experiences, such as charging satisfaction and fairness. We propose a Multi-Agent Reinforcement Learning (MARL) approach to coordinate EV charging with V2V energy exchange while considering uncertainties in the EV arrival time, energy price, and solar energy generation. The exploration capability of MARL is enhanced by introducing parameter noise into MARL's neural network models. Experimental results demonstrate the superior performance and scalability of our proposed method compared to traditional optimization baselines. The decentralized execution of the algorithm enables it to effectively deal with partial system faults in the charging station.

KW - Electric vehicle

KW - energy management

KW - multi-agent reinforcement learning (MARL)

KW - vehicle-to-grid

KW - vehicle-to-vehicle

UR - http://www.scopus.com/inward/record.url?scp=85179504838&partnerID=8YFLogxK

U2 - 10.1109/IECON51785.2023.10312315

DO - 10.1109/IECON51785.2023.10312315

M3 - Conference Paper

AN - SCOPUS:85179504838

SN - 9798350331837

T3 - IECON Proceedings (Industrial Electronics Conference)

BT - IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society

A2 - Zhu, Xing

A2 - Luque, Antonio

A2 - Chen, Weihai

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway NJ USA

T2 - Annual Conference of the IEEE Industrial Electronics Society 2023

Y2 - 16 October 2023 through 19 October 2023

ER -

Fan J, Wang H, Liebman A. MARL for decentralized electric vehicle charging coordination with V2V energy exchange. In Zhu X, Luque A, Chen W, editors, IECON 2023 - 49th Annual Conference of the IEEE Industrial Electronics Society. Piscataway NJ USA: IEEE, Institute of Electrical and Electronics Engineers. 2023. (IECON Proceedings (Industrial Electronics Conference)). doi: 10.1109/IECON51785.2023.10312315

MARL for decentralized electric vehicle charging coordination with V2V energy exchange

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Other files and links

Reliable Integration of Distributed Low-Carbon Energy Resources

Cite this

MARL for decentralized electric vehicle charging coordination with V2V energy exchange

Abstract

Publication series

Conference

Keywords

UN SDGs

Access to Document

Other files and links

Projects

Reliable Integration of Distributed Low-Carbon Energy Resources

Cite this