A Survey on Multimodal Data Stream Mining for e-Learner’s Emotion Recognition

Abstract

Emotions play a crucial role in learning. To improve and optimize electronic learning (e-Learning) outcomes, many researchers have investigated the role of emotions. Also, researchers have come up with many approaches to utilize one or many data modalities to achieve this goal, and they have been successful. But the recent advancements in technology and the internet of things (IoT) devices have brought a new dimension in e-Learning, with many input devices (such as webcams, fit-bands etc.) for interacting with e-Learners. This new dimension brings not only massive amounts of data with volume, variety, and velocity called multimodal data streams but also more challenges of mining those data in real-time. In this work, we have focused on state-of-the-art emotion recognition in e-Learning utilizing the multimodal data streams of learners. Also, we have thoroughly investigated the past research and surveys on emotion recognition methods in e-Learning to find the affecting emotions and their relations with the emotion measurement channels; and we have compared several data-stream classifiers for emotion recognition by utilizing multimodal physiological data streams. Finally, the future research opportunities to be addressed are also discussed.