Resounding Acoustic Fields with Reciprocity

2025-10-23

AI Summary

This work addresses the challenge of generalizing room impulse responses (RIRs) for dynamic sound source localization in virtual environments. We formulate the “Re-sounding” task: reconstructing RIRs at arbitrary receiver positions given only sparse, measured RIRs from known source locations. Methodologically, we introduce the first learnable physical constraint encoding acoustic reciprocity within a neural radiance field (NeRF)-based sound field representation. We propose Versa—a physics-inspired, self-supervised framework that explicitly corrects reciprocity violations induced by transceiver gain mismatches. Evaluated on both synthetic and real-world datasets, Versa significantly outperforms existing baselines. A user study confirms that its synthesized RIRs substantially enhance spatial audio immersion and directional perception accuracy. Our core contribution is the first end-to-end RIR generalization framework grounded in reciprocity priors, establishing a novel paradigm for immersive auditory modeling.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Achieving immersive auditory experiences in virtual environments requires flexible sound modeling that supports dynamic source positions. In this paper, we introduce a task called resounding, which aims to estimate room impulse responses at arbitrary emitter location from a sparse set of measured emitter positions, analogous to the relighting problem in vision. We leverage the reciprocity property and introduce Versa, a physics-inspired approach to facilitating acoustic field learning. Our method creates physically valid samples with dense virtual emitter positions by exchanging emitter and listener poses. We also identify challenges in deploying reciprocity due to emitter/listener gain patterns and propose a self-supervised learning approach to address them. Results show that Versa substantially improve the performance of acoustic field learning on both simulated and real-world datasets across different metrics. Perceptual user studies show that Versa can greatly improve the immersive spatial sound experience. Code, dataset and demo videos are available on the project website: https://waves.seas.upenn.edu/projects/versa.

Problem

• Estimating room impulse responses at arbitrary emitter positions from sparse measurements
• Leveraging acoustic reciprocity to facilitate learning of acoustic fields
• Addressing challenges in deploying reciprocity due to emitter/listener gain patterns

Innovation

• Leveraging reciprocity for acoustic field learning
• Exchanging emitter and listener poses for samples
• Self-supervised learning to address gain patterns