FlowSep 2: Self-Supervised Flow Matching for Language-Queried Audio Source Separation

2026-08-22

AI Summary

本文提出FlowSep2，通过自监督流匹配方法生成目标音频源表示，以解决自然语言查询的音频源分离问题，尤其在声音事件重叠复杂场景中表现更佳。

Abstract

Language-queried audio source separation (LASS) aims to extract target sources from audio mixtures according to natural language descriptions, offering a flexible and scalable interface for audio source separation. However, most existing LASS methods rely on discriminative, mask-based models, which estimate masks from the input mixture. These methods often over-suppress target sounds or fail to fully separate them, especially when multiple sound events strongly overlap in complex acoustic scenes. In this work, we propose FlowSep2, a text-conditioned flow-matching generative model for LASS. Instead of directly predicting a separation mask, FlowSep2 learns to generate the target source representation from Gaussian noise in a latent space, conditioned on both the mixture representation and the text query. Specifically, we employ rectified flow matching with a Diffusion Transformer backbone. We further incorporate Self-Flow, a self-supervised flow-matching paradigm, into our LASS framework. By encouraging semantically structured latent representations under the generative objective, Self-Flow improves the model's ability to separate target sources according to text queries. Experiments on multiple LASS benchmarks show that FlowSep2 achieves state-of-the-art performance and demonstrates enhanced sound separation results in challenging scenarios with overlapping sound events.

Problem

• Language-queried audio source separation
• Overlapping sound events
• Target sound separation

Innovation

• Flow Matching
• Generative Model
• Self-Supervised Learning
• Diffusion Transformer