Exploring the Design Space of Representation Learning for Audio Transformations

2026-08-28

AI Summary

研究通过处理一致性、描述对齐和前向预测等方法，解决了音频处理任务中特征表示的局限性问题。

Abstract

Neural audio representation learning has enabled a range of content-oriented applications, but the resulting features remain limited for tasks involving audio processing. Furthermore, it is not obvious what processing-aware representations should capture: the processing itself, abstracted away from source content, or the processed audio that retains it. Existing approaches implicitly commit to one or the other and also differ in their models, data, and evaluation, obscuring which design choices drive their behavior. We address both questions within a unified framework of three objectives: processing consistency, description alignment, and equivariance via forward prediction. We compare all combinations of the objectives under a controlled setup and reveal their relative strengths and interactions. Our framework produces both a transformation embedding and a processed-audio embedding, and we find that the two play complementary roles: distance-based tasks favor the former, while probe-based tasks favor the latter. Combined with improvements in network architecture and training pipeline, our representations outperform prior baselines across retrieval, probe-based evaluation, and style transfer.

Problem

• audio processing
• representation learning
• processing-aware representations

Innovation

• processing consistency
• description alignment
• equivariance via forward prediction
• transformation embedding
• processed-audio embedding