Affiliation:
1. Lero & University College Dublin, Belfield, Dublin, Ireland
2. National Instituteof Informatics, Chiyoda-ku, Tokyo, Japan
Abstract
Autonomous Driving Systems (ADSs) are promising, but must show they are secure and trustworthy before adoption. Simulation-based testing is a widely adopted approach, where the ADS is run in a simulated environment over specific scenarios. Coverage criteria specify what needs to be covered to consider the ADS sufficiently tested. However, existing criteria do not guarantee to exercise the different decisions that the ADS can make, which is essential to assess its correctness. ADSs usually compute their decisions using parameterised rule-based systems and cost functions, such as cost components or decision thresholds. In this article, we argue that the parameters characterise the decision process, as their values affect the ADS’s final decisions. Therefore, we propose parameter coverage, a criterion requiring to cover the ADS’s parameters. A scenario covers a parameter if changing its value leads to different simulation results, meaning it is relevant for the driving decisions made in the scenario. Since ADS simulators are slightly uncertain, we employ statistical methods to assess multiple simulation runs for execution difference and coverage. Experiments using the Autonomoose ADS show that the criterion discriminates between different scenarios and that the cost of computing coverage can be managed with suitable heuristics.
Funder
Science Foundation Ireland
European Regional Development Fund through the Southern & Eastern Regional Operational Programme to Lero - the Science Foundation Ireland Research Centre for Software
ERATO HASUO Metamathematics for Systems Design
Japan Society for the Promotion of Science
Engineerable AI Techniques for Practical Applications of High-Quality Machine Learning-based Systems
Research Activity Start-up
Paolo Arcaini and Fuyuki Ishikawa
Publisher
Association for Computing Machinery (ACM)
Cited by
9 articles.
订阅此论文施引文献
订阅此论文施引文献,注册后可以免费订阅5篇论文的施引文献,订阅后可以查看论文全部施引文献
1. LLM4Fin: Fully Automating LLM-Powered Test Case Generation for FinTech Software Acceptance Testing;Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis;2024-09-11
2. Industry Practices for Challenging Autonomous Driving Systems with Critical Scenarios;ACM Transactions on Software Engineering and Methodology;2024-04-18
3. Identifying and Explaining Safety-critical Scenarios for Autonomous Vehicles via Key Features;ACM Transactions on Software Engineering and Methodology;2024-04-18
4. Reality Bites: Assessing the Realism of Driving Scenarios with Large Language Models;Proceedings of the 2024 IEEE/ACM First International Conference on AI Foundation Models and Software Engineering;2024-04-14
5. Consistent and Quality-Aware Service Composition in Smart Cities;Human-Centered Services Computing for Smart Cities;2024