2019 · 14 citations · 13 references
ComputingComputer SciencePsychologyFeature EngineeringMachine LearningNatural Language ProcessingAffective ComputingDeep LearningEmotion RecognitionEmotion UnderstandingEmpathic RespondingEmpathyAffect PredictionEmotion PredictionEmotional EmpathyEmpathy ModelingFeature FusionLife Story NarrativesMultimodal Emotion RecognitionValence Modeling
People naturally understand the emotions of—and often also empathize with—those around them. In this paper, we predict the emotional valence of an empathic listener over time as they listen to a speaker narrating a life story. We use the dataset provided by the OMG-Empathy Prediction Challenge, a workshop held in conjunction with IEEE FG 2019. We present a multimodal LSTM model with feature-level fusion and local attention that predicts empathic responses from audio, text, and visual features. Our best-performing model, which used only the audio and text features, achieved a concordance correlation coefficient (CCC) of .29 and .32 on the Validation set for the Generalized and Personalized track respectively, and achieved a CCC of .14 and .14 on the held-out Test set. We discuss the difficulties faced and the lessons learnt tackling this challenge.
13
Sepp Hochreiter, Jürgen Schmidhuber · Neural Computation · 1997 · 98.1K citations
Omkar Parkhi, Andrea Vedaldi, Andrew Zisserman · 2015 · 5.1K citations · Full text