Learning Discriminative Features from Spectrograms Using Center Loss for Speech Emotion Recognition

2019-05-01

IEEE International Conference on Acoustics, Speech, and Signal Processing

AI Summary

To address the challenges of ambiguous emotional representation and weak feature discriminability in speech emotion recognition (SER), this paper introduces center loss—a metric learning technique—into SER for the first time, proposing a joint optimization framework combining softmax cross-entropy loss and center loss. The method simultaneously enhances inter-class separability and intra-class compactness on variable-length Mel-spectrograms and STFT spectrograms. Leveraging a deep convolutional neural network, it directly learns highly discriminative emotional features from raw spectrograms without handcrafted features. Experiments on standard benchmark datasets demonstrate absolute improvements of 3.2% in unweighted accuracy and 4.1% in weighted accuracy over the softmax-only baseline. This work establishes a novel paradigm for emotion feature learning in SER and empirically validates the effectiveness of metric learning for improving discriminative capability in speech-based affective computing.

Technology Category

• Machine Learning: Representation Learning
• Natural Language Processing: Speech
• Cognitive Modeling & Cognitive Systems: Affective Computing

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Identifying the emotional state from speech is essential for the natural interaction of the machine with the speaker. However, extracting effective features for emotion recognition is difficult, as emotions are ambiguous. We propose a novel approach to learn discriminative features from variable length spectrograms for emotion recognition by cooperating soft-max cross-entropy loss and center loss together. The soft-max cross-entropy loss enables features from different emotion categories separable, and center loss efficiently pulls the features belonging to the same emotion category to their center. By combining the two losses together, the discriminative power will be highly enhanced, which leads to network learning more effective features for emotion recognition. As demonstrated by the experimental results, after introducing center loss, both the unweighted accuracy and weighted accuracy are improved by over 3% on Mel-spectrogram input, and more than 4% on Short Time Fourier Transform spectrogram input.

Problem

• Speaker Emotion Recognition
• Feature Extraction
• Audio Analysis

Innovation

• softmax cross-entropy
• center loss
• affective feature learning