Source Separation by Flow Matching

2025-05-22

AI Summary

This paper addresses the ill-posed inverse problem of single-channel K-source audio separation by proposing FLOSS, the first framework to introduce flow matching into source separation. Methodologically, it designs a dimension-augmented noise injection strategy and a permutation-equivariant flow matching mechanism to jointly enforce mixture consistency and permutation invariance; employs ODE-based generative modeling, an equivariant neural architecture, and a joint constraint mapping from source to mixture distributions. Key contributions are: (1) pioneering the application of flow matching to audio source separation; and (2) unifying the resolution of dimensional mismatch and permutation ambiguity via dimension augmentation and equivariance. Evaluated on overlapping speech separation, FLOSS achieves significant improvements in reconstruction fidelity and mixture consistency over prior methods.

Technology Category

• Machine Learning: Multimodal Learning
• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We consider the problem of single-channel audio source separation with the goal of reconstructing $K$ sources from their mixture. We address this ill-posed problem with FLOSS (FLOw matching for Source Separation), a constrained generation method based on flow matching, ensuring strict mixture consistency. Flow matching is a general methodology that, when given samples from two probability distributions defined on the same space, learns an ordinary differential equation to output a sample from one of the distributions when provided with a sample from the other. In our context, we have access to samples from the joint distribution of $K$ sources and so the corresponding samples from the lower-dimensional distribution of their mixture. To apply flow matching, we augment these mixture samples with artificial noise components to ensure the resulting"augmented"distribution matches the dimensionality of the $K$ source distribution. Additionally, as any permutation of the sources yields the same mixture, we adopt an equivariant formulation of flow matching which relies on a suitable custom-designed neural network architecture. We demonstrate the performance of the method for the separation of overlapping speech.

Problem

• Single-channel audio source separation for reconstructing K sources
• Ensuring mixture consistency via constrained flow matching (FLOSS)
• Handling source permutation ambiguity with equivariant flow matching

Innovation

• Flow matching for source separation (FLOSS)
• Augmented mixture samples with artificial noise
• Equivariant flow matching with custom neural network