Learnable Adaptive Time-Frequency Representation via Differentiable Short-Time Fourier Transform

2025-06-26

AI Summary

Conventional short-time Fourier transform (STFT) performance critically depends on hand-crafted or heuristic hyperparameters—such as window length, hop size, and overlap ratio—while existing discrete grid-search optimization methods suffer from high computational cost and lack task-specific adaptability. Method: We propose the first fully differentiable STFT framework, modeling core STFT operations—including window function selection, overlap ratio, and discrete Fourier transform—as end-to-end trainable, differentiable signal processing modules. This enables joint optimization with downstream neural networks via gradient-based learning. Contribution/Results: By transcending the limitations of discrete parameter spaces, our approach achieves gradient-driven, adaptive time-frequency representation learning. Extensive experiments on synthetic and real-world signals demonstrate substantial improvements in time-frequency resolution and consistent performance gains across downstream tasks—including classification and denoising—validating both the effectiveness and generalizability of differentiable time-frequency analysis.

Technology Category

• Machine Learning: Time-Series/Data Streams
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

The short-time Fourier transform (STFT) is widely used for analyzing non-stationary signals. However, its performance is highly sensitive to its parameters, and manual or heuristic tuning often yields suboptimal results. To overcome this limitation, we propose a unified differentiable formulation of the STFT that enables gradient-based optimization of its parameters. This approach addresses the limitations of traditional STFT parameter tuning methods, which often rely on computationally intensive discrete searches. It enables fine-tuning of the time-frequency representation (TFR) based on any desired criterion. Moreover, our approach integrates seamlessly with neural networks, allowing joint optimization of the STFT parameters and network weights. The efficacy of the proposed differentiable STFT in enhancing TFRs and improving performance in downstream tasks is demonstrated through experiments on both simulated and real-world data.

Problem

• Optimizing STFT parameters via differentiable formulation
• Enhancing time-frequency representation with gradient-based tuning
• Integrating STFT optimization with neural network training

Innovation

• Differentiable STFT for gradient-based parameter optimization
• Seamless integration with neural network joint training
• Enhanced time-frequency representation for downstream tasks