DEAP: A Database for Emotion Analysis ;Using Physiological Signals

TLDR

The study introduces a multimodal dataset for human affective state analysis and proposes a novel stimuli‑selection method based on affective tags, video highlights, and an online assessment tool. The dataset was collected by recording EEG, peripheral physiological signals, and (for some participants) frontal face video while 32 participants watched 40 one‑minute music‑video excerpts and rated affective dimensions; the study then examined EEG‑rating correlations, performed single‑trial classification of arousal, valence, and like/dislike using multimodal data, and fused modality‑specific classifiers. The study provides an extensive analysis of participants’ ratings, presents single‑trial classification results for arousal, valence, and like/dislike using EEG, peripheral signals, and multimedia content, and releases the dataset publicly for use by other researchers.

Abstract

We present a multimodal data set for the analysis of human affective states. The electroencephalogram (EEG) and peripheral physiological signals of 32 participants were recorded as each watched 40 one-minute long excerpts of music videos. Participants rated each video in terms of the levels of arousal, valence, like/dislike, dominance, and familiarity. For 22 of the 32 participants, frontal face video was also recorded. A novel method for stimuli selection is proposed using retrieval by affective tags from the last.fm website, video highlight detection, and an online assessment tool. An extensive analysis of the participants' ratings during the experiment is presented. Correlates between the EEG signal frequencies and the participants' ratings are investigated. Methods and results are presented for single-trial classification of arousal, valence, and like/dislike ratings using the modalities of EEG, peripheral physiological signals, and multimedia content analysis. Finally, decision fusion of the classification results from different modalities is performed. The data set is made publicly available and we encourage other researchers to use it for testing their own affective state estimation methods.