Driver Drowsiness Detection based on Multimodal using Fusion of Visual-feature and Bio-signal

Abstract

It is very dangerous for a driver to fall into a momentary drowsiness. Previous studies to prevent this have classified the driver's condition mainly by using features that appear when drowsiness occurs. At this time, there is insufficient information to judge the driver's condition by using only single data (physiological data, visual data). In this study, we propose a system based on Multimodal Deep Learning that recognizes both visual and physiological changes in drowsiness. Because using different kind of data, heterogeneity problem arise. So in order to eliminate heterogeneity between data, using generative model to representation. Since drowsiness is a change that occurs with time, we use a deep learning network consisting of Long Short-Term Memory (LSTM) to classify the driver's condition.