Weakly Guided and Autoregressive Beamformer Parameterization for Generalizable Moving Speaker Extraction in Higher-Order Ambisonics

2026-07-05

AI Summary

Existing deep beamforming approaches exhibit limited generalization in scenarios involving multiple dynamically moving speakers with unknown directions. This work proposes a weakly guided, autoregressive beamforming framework that requires only an initial directional estimate of the target speaker. By operating in higher-order Ambisonics representation, the method decouples neural time–frequency modeling from linear spatial filtering, enabling microphone-array-agnostic speech enhancement. A frame-level causal autoregressive mechanism is introduced—the first of its kind in causal beamforming—to facilitate continuous tracking of rapidly moving speakers. The approach demonstrates robust enhancement performance on synthetic data featuring closely spaced, interleaved moving speakers and shows strong cross-array and cross-Ambisonics-order generalization on real-world office meeting recordings.

Abstract

Linear spatial filters (beamformers) enable robust, generalizable and interpretable speech enhancement with performance guarantees under ideal parameterization. Modern beamformers are often parameterized by deep neural networks, whose performance degrades in dynamic scenarios with multiple moving speakers of unknown directions. We propose a data-driven beamforming pipeline, which only requires an estimate of the target's initial direction. Building on a higher-order ambisonics representation, we show that neural temporal-spectral processing can be decoupled from linear spatial processing, and thereby achieve generalizable and array-agnostic enhancement. By incorporating autoregression into a frame-wise causal framework, we maintain consistent performance throughout fast speaker motion and long recordings. Evaluation on synthetic data demonstrates robust enhancement under challenging conditions with closely spaced and crossing speakers. Real-world recordings in a dynamic office meeting scenario complement these findings and show generalizability across varying ambisonics orders.

Problem

• moving speaker extraction
• higher-order Ambisonics
• beamformer generalization
• dynamic speech scenarios
• unknown speaker directions

Innovation

• autoregressive beamforming
• higher-order Ambisonics
• moving speaker extraction
• array-agnostic enhancement
• weakly guided parameterization