A Bayesian Deep Learning Framework for End-To-End Prediction of Emotion From Heartbeat

Abstract

Automatic prediction of emotion promises to revolutionise human-computer interaction. Recent trends involve fusion of multiple data modalities <inline-formula><tex-math notation="LaTeX">$-$</tex-math><alternatives><mml:math><mml:mo>-</mml:mo></mml:math><inline-graphic xlink:href="harper-ieq1-2981610.gif"/></alternatives></inline-formula> audio, visual, and physiological <inline-formula><tex-math notation="LaTeX">$-$</tex-math><alternatives><mml:math><mml:mo>-</mml:mo></mml:math><inline-graphic xlink:href="harper-ieq2-2981610.gif"/></alternatives></inline-formula> to classify emotional state. However, in practice, collection of physiological data &#x2018;in the wild&#x2019; is currently limited to heartbeat time series of the kind generated by affordable wearable heart monitors. Furthermore, real-world applications of emotion prediction often require some measure of uncertainty over model output, in order to inform downstream decision-making. We present here an end-to-end deep learning model for classifying emotional valence from unimodal heartbeat time series. We further propose a Bayesian framework for modelling uncertainty over these valence predictions, and describe a probabilistic procedure for choosing to accept or reject model output according to the intended application. We benchmarked our framework against two established datasets and achieved peak classification accuracy of 90 percent. These results lay the foundation for applications of affective computing in real-world domains such as healthcare, where a high premium is placed on non-invasive collection of data, and predictive certainty.