Feature Analysis and Evaluation for Automatic Emotion Identification in Speech

TLDR

Identifying emotions in speech requires careful feature definition, yet no consensus exists on the optimal features; prosody is widely used, often alongside spectral and voice‑quality parameters, but systematic comparisons are lacking. The study analyzes how prosodic, spectral envelope, and voice‑quality features discriminate emotions. The authors evaluate early and late fusion of prosodic, spectral envelope, and voice‑quality features, validating the approach with automatic emotion identification experiments. Spectral envelope features outperform prosodic ones, and late fusion of long‑term spectral statistics with short‑term spectral envelope parameters achieves accuracy comparable to combining all feature types.

Abstract

The definition of parameters is a crucial step in the development of a system for identifying emotions in speech. Although there is no agreement on which are the best features for this task, it is generally accepted that prosody carries most of the emotional information. Most works in the field use some kind of prosodic features, often in combination with spectral and voice quality parametrizations. Nevertheless, no systematic study has been done comparing these features. This paper presents the analysis of the characteristics of features derived from prosody, spectral envelope, and voice quality as well as their capability to discriminate emotions. In addition, early fusion and late fusion techniques for combining different information sources are evaluated. The results of this analysis are validated with experimental automatic emotion identification tests. Results suggest that spectral envelope features outperform the prosodic ones. Even when different parametrizations are combined, the late fusion of long-term spectral statistics with short-term spectral envelope parameters provides an accuracy comparable to that obtained when all parametrizations are combined.