Dual-BEATs: Unlocking Zero-Shot Stereo Audio Perception in Audio Large Language Models via Dithering

2026-07-08

AI Summary

Existing multimodal large language models lack spatial awareness due to their reliance on monaural audio representations, and current approaches often require complex room simulations or custom stereo encoders, limiting their generalizability. This work proposes Dual-BEATs, a novel architecture that enables zero-shot spatial perception of stereophonic audio in standard multimodal models without dedicated spatial modules or additional training. By employing dual-channel independent semantic encoding, injecting static uncorrelated jitter noise at the input stage, and preserving inter-channel variance through normalization layers, Dual-BEATs achieves 97.2% accuracy on left/center/right localization tasks and remains highly effective even for subtle panning shifts as small as 0.5. Furthermore, the method demonstrates strong zero-shot generalization across diverse spatial configurations.

Abstract

Multimodal Large Language Models (LLMs) have remarkable semantic audio understanding, yet they remain "spatially agnostic" due to their reliance on mono-channel audio representations. Currently, spatial audio perception methods mainly focus on complex room simulations and custom-trained, geometry-aware stereo encoders, which limits their accessibility and generalizability. In this paper, we introduce the Dual-BEATs architecture, in which the left and right audio channels are routed independently through two identical semantic encoders as an alternative to specialized spatial modules. To circumvent the architectural bottleneck where internal normalization otherwise erases the inter-channel variance of stereo audio, we inject a static, uncorrelated dithering noise floor prior to encoding. This dithering intervention establishes a macro-variance floor that "smuggles" spatial geometry across the normalization layers. Evaluated on a ternary directional classification task (Left, Center, Right), we demonstrate that dithered models achieve exceptional spatial resolution--reaching up to 97.2% localization accuracy even on subtle 0.5 panning amplitudes--and demonstrates robust, zero-shot generalization to entirely unseen spatial configurations. Our results suggest that with the appropriate acoustic regularization, standard multimodal models are natively capable of generalized stereo audio understanding.

Problem

• spatial audio perception
• multimodal LLMs
• stereo audio
• zero-shot generalization
• audio representation

Innovation

• Dual-BEATs
• dithering
• zero-shot stereo perception
• spatial audio understanding
• multimodal LLMs