Deep Transfer Learning Enable End-to-End Steering Angles Prediction for Self-driving Car

Abstract

Autonomous driving has developed rapidly over the last few years. Predicting the steering angle for self-driving car according to different road conditions is very important. There are some endeavors for this topic, including lane detection, object detection on roads, 3-D reconstruction etc., but in our work we focus on a vision based model that directly maps raw input images to steering angles using deep networks and this model don't depend on specifying the features to learn. In this paper, we propose an end-to-end steering angle prediction model based on deep transfer learning and it can accurately predicts steering angles based on input image sequences which are from onboard camera. This prediction model combine two deep learning models including the convolution neural network (CNN) and the long short-term memory (LSTM). The CNN model we use is VGG16 which is based on transfer learning techniques, and pre-trained on Imagenet with good performance. This network is used to extract spatial features of the input image sequences. And the LSTM network is used to capture the temporal information of the provided images. The model we proposed fully considers spatial-temporal information, and fit the nonlinear relationship well between the input images and the steering angles. In order to validate the proposed model, the experimental study is conducted using the real-world dataset which is provided by Udacity. Experimental results show that the proposed model in this paper can efficiently predict the steering angles and clone humans' driving behaviors, and our model has a better performance, higher accuracy, and less training time.