Spectral and Rhythm Feature Performance Evaluation for Category and Class Level Audio Classification with Deep Convolutional Neural Networks

2025-09-09

AI Summary

This study systematically evaluates the impact of six time-frequency representations—Mel spectrograms, MFCCs, STFT chromagrams, CQT chromagrams, CENS chromagrams, and cyclic tempograms—on environmental audio classification performance under a unified deep CNN architecture. Using the ESC-50 dataset, all features are trained end-to-end under identical experimental conditions, and their performance is rigorously compared across both coarse-grained (category-level) and fine-grained classification tasks using accuracy, precision, recall, and F1-score. Results demonstrate that Mel spectrograms and MFCCs significantly outperform the other representations, achieving top overall accuracies of 86.2% and 85.7%, respectively—confirming their robustness and discriminative power for modeling environmental acoustic structure. To our knowledge, this is the first work to conduct a controlled, cross-representation benchmark under consistent network architecture and training protocol. The findings provide reproducible, empirical guidance for feature selection in environmental audio analysis.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Cognitive Modeling & Cognitive Systems: Symbolic Representations

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Next to decision tree and k-nearest neighbours algorithms deep convolutional neural networks (CNNs) are widely used to classify audio data in many domains like music, speech or environmental sounds. To train a specific CNN various spectral and rhythm features like mel-scaled spectrograms, mel-frequency cepstral coefficients (MFCC), cyclic tempograms, short-time Fourier transform (STFT) chromagrams, constant-Q transform (CQT) chromagrams and chroma energy normalized statistics (CENS) chromagrams can be used as digital image input data for the neural network. The performance of these spectral and rhythm features for audio category level as well as audio class level classification is investigated in detail with a deep CNN and the ESC-50 dataset with 2,000 labeled environmental audio recordings using an end-to-end deep learning pipeline. The evaluated metrics accuracy, precision, recall and F1 score for multiclass classification clearly show that the mel-scaled spectrograms and the mel-frequency cepstral coefficients (MFCC) perform significantly better then the other spectral and rhythm features investigated in this research for audio classification tasks using deep CNNs.

Problem

• Evaluating spectral and rhythm features for audio classification
• Comparing feature performance using deep convolutional neural networks
• Assessing MFCC and mel-spectrograms against alternative audio representations

Innovation

• Deep convolutional neural networks for audio classification
• Evaluating spectral and rhythm features as image inputs
• Mel-spectrograms and MFCCs outperform other features