Real-time Object Detection in Autonomous Vehicles with YOLO

Abstract

AI analytics enables autonomous cars to detect and recognize objects, such as other vehicles, pedestrians, traffic signs, and obsta- cles, in real-time. Deep learning models, notably the You Only Look Once (YOLO) model, have demonstrated accuracy and speed in obstacle avoidance. However, current datasets are limited, lacking diversity and labeling, hindering their ability to represent real- world scenarios accurately. Besides, previous studies have focused extensively on specific object classes, such as pedestrians and vehicles, often neglecting other objects like bikes and road signs. To address this, we introduce a novel dataset tailored for AV envi- ronments, encompassing various road object types under different conditions. Our innovative methodology relies on self-supervised learning using the late YOLO version to improve model robustness with limited labeled data and AI-driven adaptive model opti- mization based on real-time feedback. We evaluate three YOLO architectures—YOLOv5, YOLOv7, and YOLOv8—customized for AV object detection. Our assessment covers everyday AV objects such as cars, pedestrians, bicycles, and road signs, empha- sizing early detection. We employ the VSim-AV simulator dataset to ensure robust evaluation, augmented with preprocessing techniques to optimize data quality and model generalization. The study reveals that YOLOv5 and YOLOv8 outperform YOLOv7 regarding precision and recall across various object classes, with YOLOv5 leading at 1.3 ms/image and YOLOv8 at 3.3 ms/image. The mean average precision was 0.94 for YOLOv5, 0.441 for YOLOv7, and 0.927 for YOLOv8, highlighting the limitations in current literature and challenges in YOLO model performance.