AnyGoal: Vision-Language Guided Multi-Agent Exploration for Training-Free Lifelong Navigation

2026-06-11

AI Summary

This work addresses the poor out-of-distribution generalization of end-to-end navigation and the limited efficiency and scalability of existing modular or 3D memory-based systems by proposing a training-free multi-agent navigation framework. The approach uniquely integrates a vision-language model (VLM) with a shared 2D Gaussian Bayesian Value Map (BVM), leveraging depth-cone masking fusion and an upper confidence bound (UCB)-based exploration strategy to enable lifelong exploration. It further introduces an evidence accumulation mechanism and a greedy task allocator with spatial separation penalties to jointly perform frontier selection and task assignment. Evaluated on GOAT-Bench, the two-agent system achieves a subtask success rate of 52.4%, surpassing Modular GOAT by 27.5 percentage points, while the single-agent variant attains 41.9%, demonstrating strong effectiveness and generalization in open-vocabulary object navigation.

Abstract

End-to-end navigation policies trained on large simulation corpora degrade sharply when transferred to out-of-distribution scenes, categories, or goal modalities. Modular pipelines such as Modular GOAT are bottlenecked by closed-set object detection recall, while 3D snapshot-memory systems (e.g. 3D-Mem) accumulate dense, view-dependent representations that are heavy to maintain. We present AnyGoal, a training-free multi-robot architecture that places a Vision-Language Model (VLM) at the core of frontier-based exploration and coordinates agents through a shared 2D Gaussian Bayesian Value Map (BVM). The BVM maintains a per-pixel (mu, sigma^2) posterior over goal relevance, updated via precision-weighted fusion of VLM scores through a depth-cone mask, and is never reset between subtasks, yielding lifelong evidence accumulation. Frontiers are ranked by a convex blend of a VLM-as-judge softmax and a Bayesian UCB term on the BVM. A greedy allocator with spatial-separation penalty and commitment hysteresis distributes frontiers across agents without a centralized controller. On the full GOAT-Bench val unseen split (360 episodes, 2,669 subtasks), our dual-agent system achieves 52.4% Subtask SR at 12.7% SPL--state of the art under the strict physical regime (discrete 0.25 m steps, no teleportation, 42 deg HFOV) and a +27.5 pp improvement over Modular GOAT (24.9%). Single-agent AnyGoal achieves 41.9% Subtask SR, showing gains arise from the decision architecture. A four-way perception ablation shows that open-vocabulary detectors shift the dominant failure mode from exploration to goal verification.

Problem

• lifelong navigation
• out-of-distribution generalization
• multi-agent exploration
• vision-language grounding
• training-free navigation

Innovation

• Vision-Language Model
• Bayesian Value Map
• Training-Free Navigation
• Multi-Agent Exploration
• Lifelong Navigation