Off-Road Obstacle Avoidance through End-to-End Learning

TLDR

The authors present a vision‑based obstacle‑avoidance system for off‑road mobile robots. The system is trained end‑to‑end with a 6‑layer convolutional network that maps raw low‑resolution images from two forward‑pointing cameras to steering angles, using supervised data collected from a human driver across diverse terrains, weather, lighting, and obstacle types, with a remote computer processing the video and controlling the 50 cm truck via radio. The trained robot reliably detects obstacles and navigates around them in real time at 2 m/s.

Abstract

We describe a vision-based obstacle avoidance system for off-road mobile robots. The system is trained from end to end to map raw input images to steering angles. It is trained in supervised mode to predict the steering angles provided by a human driver during training runs collected in a wide variety of terrains, weather conditions, lighting conditions, and obstacle types. The robot is a 50cm off-road truck, with two forward-pointing wireless color cameras. A remote computer processes the video and controls the robot via radio. The learning system is a large 6-layer convolutional network whose input is a single left/right pair of unprocessed low-resolution images. The robot exhibits an excellent ability to detect obstacles and navigate around them in real time at speeds of 2 m/s.