Read my lips, login to the virtual world

Abstract

Login systems in smart devices demand multi-factor authentication for high security and at the same time, it requires simple user experience. We propose a novel application of lip-reading satisfying these requirements. We present the adequacy of lip-reading as a biometric factor by experiment. In addition, automatic lip-reader can be implemented by LSTM (Long Short Term Memory) neural network architecture with good accuracy that can translate visual utterance to password as a knowledge factor. Furthermore, our proposed method, iterative method, can improve accuracy as much as login system required. Our work achieved 93.8% by single iteration from the first result (69.1%).