Affiliation:
1. China Communications Information & Technology Group Co., Ltd., Beijing 101399, China
2. State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing 100876, China
3. Talent Exchange Center, Ministry of Industry and Information Technology, Beijing 100846, China
Abstract
The proliferation of wireless technologies, particularly the advent of 5G networks, has ushered in transformative possibilities for enhancing vehicular communication systems, particularly in the context of autonomous driving. Leveraging sensory data and mapping information downloaded from base stations using I2V links, autonomous vehicles in these networks present the promise of enabling distant perceptual abilities essential to completing various tasks in a dynamic environment. However, the efficient down-link transmission of vehicular network data via base stations, often relying on spectrum sharing, presents a multifaceted challenge. This paper addresses the intricacies of spectrum allocation in vehicular networks, aiming to resolve the thorny issues of cross-station interference and coupling while adapting to the dynamic and evolving characteristics of the vehicular environment. A novel approach is suggested involving the utilization of a multi-agent option-critic reinforcement learning algorithm. This algorithm serves a dual purpose: firstly, it learns the most efficient way to allocate spectrum resources optimally. Secondly, it adapts to the ever-changing dynamics of the environment by learning various policy options tailored to different situations. Moreover, it identifies the conditions under which a switch between these policy options is warranted as the situation evolves. The proposed algorithm is structured in two layers, with the upper layer consisting of policy options that are shared across all agents, and the lower layer comprising intra-option policies executed in a distributed manner. Through experimentation, we showcase the superior spectrum efficiency and communication quality achieved by our approach. Specifically, our approach outperforms the baseline methods in terms of training average reward convergence stability and the transmission success rate. Control-variable experiments also reflect the better adaptability of the proposed method as the environmental conditions change, underscoring the significant potential of the proposed method in aiding successful down-link transmissions in vehicular networks.
Funder
National Key Research and Development Program of China
BUPT Innovation and Entrepreneurship Support Program
Reference44 articles.
1. Sun, J., Fang, X., and Zhang, Q. (2023, January 12–14). Reinforcement Learning Driving Strategy Based on Auxiliary Task for Multi-Scenarios Autonomous Driving. Proceedings of the 2023 IEEE 12th Data Driven Control and Learning Systems Conference (DDCLS), Xiangtan, China.
2. Mishra, A., Purohit, J., Nizam, M., and Gawre, S.K. (2023, January 18–19). Recent Advancement in Autonomous Vehicle and Driver Assistance Systems. Proceedings of the 2023 IEEE International Students’ Conference on Electrical, Electronics and Computer Science (SCEECS), Bhopal, India.
3. Recent Advancements in End-To-End Autonomous Driving Using Deep Learning: A Survey;Chib;IEEE Trans. Intell. Veh.,2023
4. Huang, Y., Chen, Y., and Yang, Z. (2023). An Overview about Emerging Technologies of Autonomous Driving. arXiv.
5. Advancements and Challenges in Achieving Fully Autonomous Self-Driving Vehicles;Kosuru;World J. Adv. Res. Rev.,2023