Beyond Dry References: Learning Relative Audio Effects Representations via Contrastive Distance Learning

2026-08-11

AI Summary

This work addresses the limitation of existing audio effect modeling approaches, which rely on clean dry references that are often unavailable in real-world recordings due to uncontrolled factors such as microphone characteristics and room acoustics. To overcome this, the authors propose RelFx, a novel framework that introduces the concept of relative effect distance for the first time. RelFx employs a Siamese dual-branch encoder enhanced with cross-attention and differential gating fusion, along with an antisymmetric fusion mechanism to enable bidirectional effect embeddings—allowing the model to learn relative effect transformations between audio signals without requiring dry references. Evaluated on the Fx style transfer task under the standard MUSDB18 protocol, the method achieves state-of-the-art performance, significantly outperforming existing approaches across all four instrument categories.

Abstract

Audio effects (Fx) representation learning plays a key role in intelligent music production, including automatic mixing and Fx style transfer. Existing methods typically rely on dry or nearly dry references for effect modeling, yet truly unprocessed audio is rarely available in practice, as real recordings inevitably reflect the microphone, room acoustics, and preceding signal processing. Instead of pursuing absolute effect encodings, we argue that the relative effect distance between audio signals is more meaningful for real-world music production. Motivated by this, we propose RelFx, a contrastive learning framework that learns relative effect transformations from general audio collections without requiring dry references during representation training. Our approach uses a dual-branch Siamese encoder equipped with cross-attention and differential gating fusion to infer the shared effect transformation from a reference clip and an effect-processed, content-related clip. We further propose an antisymmetric fusion variant for bidirectional effect encoding, such that swapping the input order directly produces a nearly sign-reversed embedding, a property not explored in earlier work. Moreover, our dry-reference-free formulation eliminates the reliance on dry multitrack datasets and enables training on effect-bearing audio. Experiments on Fx style transfer demonstrate state-of-the-art performance under the standard Fx-Encoder++ MUSDB18 evaluation protocol, consistently outperforming existing approaches across all four instrument categories.

Problem

• audio effects
• representation learning
• dry reference
• relative effect distance
• music production

Innovation

• relative audio effects
• contrastive learning
• dry-reference-free
• Siamese encoder
• antisymmetric fusion