Convolutional Neural Networks for Predicting Words: A Lip-Reading System

PV Sindhura, S. Preethi, Krupa B Niranjana

2018 International Conference on Electrical, Electronics, Communication, Computer, and Optimization Techniques (ICEECCOT) · 2018 · 15 citations · 13 references

Concepts

Abstract

In this paper, a relative study of lip reading models based on convolutional neural networks (CNN), like AlexNet and Inception V3, has been carried out. The novelty of the paper lies in studying the performance of speaker dependent and speaker independent models, in predicting ten words, making use of the two aforementioned CNNs. In this process a dataset consisting of ten words, fifteen speakers' data named Miracl- VC1 has been used. From all the samples lip regions are extracted and concatenated for every word and used in training. Both the models are tested for known (speaker dependent) and unknown speakers (speaker independent). AlexNet reported the highest observed test accuracy of 86.6%, and Inception V3 reported an accuracy of 64.6% for speaker dependent Lip reading. Whereas, speaker independent test accuracies of both the models are at 37.1% in case of AlexNet and 17.6% for Inception-V3.

References

13