Towards Explainable Music Emotion Recognition: The Route via Mid-level\n Features

Shreyan Chowdhury, Andreu Vall, Verena Haunschmid, Gerhard Widmer

arXiv (Cornell University) · 2019 · 13 citations · 0 references

DOIFull text

Open access

Concepts

Abstract

Emotional aspects play an important part in our interaction with music.\nHowever, modelling these aspects in MIR systems have been notoriously\nchallenging since emotion is an inherently abstract and subjective experience,\nthus making it difficult to quantify or predict in the first place, and to make\nsense of the predictions in the next. In an attempt to create a model that can\ngive a musically meaningful and intuitive explanation for its predictions, we\npropose a VGG-style deep neural network that learns to predict emotional\ncharacteristics of a musical piece together with (and based on)\nhuman-interpretable, mid-level perceptual features. We compare this to\npredicting emotion directly with an identical network that does not take into\naccount the mid-level features and observe that the loss in predictive\nperformance of going through the mid-level features is surprisingly low, on\naverage. The design of our network allows us to visualize the effects of\nperceptual features on individual emotion predictions, and we argue that the\nsmall loss in performance in going through the mid-level features is justified\nby the gain in explainability of the predictions.\n