VCNAC: A Variable-Channel Neural Audio Codec for Mono, Stereo, and Surround Sound

2026-01-21

AI Summary

This work addresses the challenge that conventional neural audio codecs struggle to efficiently support multiple channel configurations—such as mono, stereo, and 5.1 surround sound—within a single model while lacking cross-channel compatibility. The authors propose a variable-channel neural audio codec that, for the first time, natively supports diverse channel layouts through a shared-parameter encoder-decoder architecture and a unified codebook representation. Crucially, they introduce a channel compatibility optimization objective to preserve perceptual quality when higher-channel content is rendered on lower-channel playback systems. Experimental results demonstrate that the proposed method achieves high-fidelity reconstruction in both objective spatial audio metrics and subjective listening tests, while enabling flexible, on-the-fly channel scalability during inference.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search

Abstract

We present VCNAC, a variable channel neural audio codec. Our approach features a single encoder and decoder parametrization that enables native inference for different channel setups, from mono speech to cinematic 5.1 channel surround audio. Channel compatibility objectives ensure that multi-channel content maintains perceptual quality when decoded to fewer channels. The shared representation enables training of generative language models on a single set of codebooks while supporting inference-time scalability across modalities and channel configurations. Evaluation using objective spatial audio metrics and subjective listening tests demonstrates that our unified approach maintains high reconstruction quality across mono, stereo, and surround audio configurations.

Problem

• neural audio codec
• variable-channel
• channel compatibility
• spatial audio
• unified representation

Innovation

• variable-channel codec
• neural audio coding
• channel compatibility
• shared representation
• spatial audio