LH-AVLN: A Benchmark for Long-Horizon Audio-Visual-Language Navigation

2026-07-04

AI Summary

Existing long-horizon navigation benchmarks lack audio cues, and current audio-visual navigation approaches are largely confined to single-target settings with unimodal instructions, struggling to handle complex, multi-target tasks involving heterogeneous modalities. This work introduces the first audio-visual language navigation benchmark that integrates spatialized audio, heterogeneous target specifications (category labels, language descriptions, and images), and long-range multi-target navigation, supporting both ordered and unordered task structures. Crucially, it treats sound as both a non-line-of-sight guidance signal and a potential source of interference, introducing novel challenges. Leveraging RGB-D, binaural audio, and pose inputs, we propose PAG-Nav, a training-free agent that performs multi-stage navigation through a temporally consistent semantic map and progressive goal planning. Experiments reveal limited performance of existing agents, while PAG-Nav establishes a strong diagnostic baseline, highlighting substantial room for improvement in this challenging setting.

Abstract

Embodied navigation is moving toward long-horizon missions, yet existing long-horizon benchmarks are largely acoustically silent, and audio-visual navigation tasks typically focus on a single goal. We introduce LH-AVLN, a benchmark for Long-Horizon Audio-Visual-Language Navigation that combines multi-goal mission execution, heterogeneous goal specifications, and persistent spatialized acoustic cues. In LH-AVLN, an agent receives a global mission of two to four goals specified by category, language description, or reference image, and navigates with RGB-D observations, pose, and binaural audio in indoor 3D environments. The benchmark supports both ordered and unordered missions, where alternating goal-associated sounds can guide non-line-of-sight search but may also become distractors as mission progress changes. We further develop PAG-Nav, a training-free reference agent that maintains a temporal uniform semantic map and performs progressive goal-state planning, using sound for search while reserving completion for visual-semantic verification. Experiments show that existing vision-language, memory-based, and audio-visual agents struggle to complete full LH-AVLN missions, and that PAG-Nav provides a stronger diagnostic baseline while leaving substantial room for future progress.

Problem

• long-horizon navigation
• audio-visual-language navigation
• multi-goal missions
• spatialized audio
• embodied AI

Innovation

• Long-Horizon Navigation
• Audio-Visual-Language Navigation
• Spatialized Audio
• Multi-Goal Mission
• Training-Free Agent