Explainable Object Detection for Uncrewed Aerial Vehicles using KernelSHAP

Abstract

While the field of object detection has seen remarkable performance gains since the incorporation of Deep Neural Networks (DNNs), a significant drawback in DNN detection algorithms is that they lack transparency, making their behaviour somewhat unpredictable. Without transparency, employing DNNs for on-board object detection on Uncrewed Aerial Vehicles (UAVs) could have massive societal and safety consequences. In this paper, we propose adopting a proven explainer, KernelSHAP, to provide visual explanations for bounding boxes produced by detection algorithms intended for on-board UAVs. Our explainer can identify the important parts of an image that assisted a given detection or contributed to a specific failure mode. We evaluate our explainer’s discriminative ability on aerial imagery through a pointing metric and an automatic deletion/insertion metric. We further assess our explainer by intentionally introducing a bias to the dataset for it to detect and using that bias to simulate failure modes that can then be discovered using our explainer.