Virtual Tools for Testing Autonomous Driving: A Survey and Benchmark of Simulators, Datasets, and Competitions

Abstract

Traditional road testing of autonomous vehicles faces significant limitations, including long testing cycles, high costs, and substantial risks. Consequently, autonomous driving simulators and dataset-based testing methods have gained attention for their efficiency, low cost, and reduced risk. Simulators can efficiently test extreme scenarios and provide quick feedback, while datasets offer valuable real-world driving data for algorithm training and optimization. However, existing research often provides brief and limited overviews of simulators and datasets. Additionally, while the role of virtual autonomous driving competitions in advancing autonomous driving technology is recognized, comprehensive surveys on these competitions are scarce. This survey paper addresses these gaps by presenting an in-depth analysis of 22 mainstream autonomous driving simulators, focusing on their accessibility, physics engines, and rendering engines. It also compiles 35 open-source datasets, detailing key features in scenes and data-collecting sensors. Furthermore, the paper surveys 10 notable virtual competitions, highlighting essential information on the involved simulators, datasets, and tested scenarios involved. Additionally, this review analyzes the challenges in developing autonomous driving simulators, datasets, and virtual competitions. The aim is to provide researchers with a comprehensive perspective, aiding in the selection of suitable tools and resources to advance autonomous driving technology and its commercial implementation.