From Collision to Verdict: Responsibility Attribution for Autonomous Driving Systems Testing

Abstract

Autonomous driving systems (ADS) are safety-critical systems that require thorough testing to ensure their safety. Current testing methods for ADS primarily focus on finding crash scenarios involving ADS. However, most of these scenarios are unavoidable by ADS, such as collisions caused by the reckless behavior of other vehicles. To address this limitation, we propose CollVer, a framework designed to generate and identify scenarios in which ADS violate driving rules. Specifically, CollVer utilizes multi-modal technology by taking the violation scenario and the corresponding accident description as inputs to judge whether the accident can be attributed to the ADS. Moreover, CollVer introduces a metric called collision position coverage (CPC), to quantify and guide the selection of test cases. Finally, CollVer integrates the multi-modal model and the CPC metric into a multi-objective genetic algorithm to explore more diverse and challenging scenarios. We evaluate CollVer on an industrial-grade ADS, Baidu Apollo, and experimental results show that CollVer can identify 10 distinct types of safety violations, with 4 of them resulting from ADS violating driving rules.