VerNav: Verifier-First Low-Latency Vision-and-Language Navigation

2026-09-01

AI Summary

VerNav通过引入验证器优先框架和两阶段对齐方案减少基于语言模型的视觉-语言导航中的决策延迟，同时保持良好的导航性能。

Abstract

Vision-and-Language Navigation (VLN) requires an agent to navigate through unseen 3D environments according to natural-language instructions. Explicit reasoning can improve instruction understanding and semantic grounding, but autoregressive generation at every step accumulates large decision-stage latency over multi-step navigation. We propose VerNav, a verifier-first framework for low-latency LLM-based VLN. The verifier reduces decision-stage latency by replacing per-step autoregressive generation with batched action verification, while an entropy-based adaptive generator is invoked only for uncertain decisions to produce compact state evidence. To further improve navigation performance with the verifier, we introduce a two-stage alignment scheme: (i) VPO improves local action-preference alignment in static verifier training, and (ii) step-level reinforcement fine-tuning provides dense progress rewards over multi-step navigation rollouts during dynamic task execution. Experiments on the Room-to-Room (R2R) benchmark show that the verifier-only decision path of VerNav achieves competitive navigation performance among representative LLM-based VLN agents while reducing average decision-stage LLM latency per step by more than $10\times$ compared with autoregressive methods.

Problem

• Vision-and-Language Navigation
• low-latency
• autoregressive generation

Innovation

• Verifier-first framework
• Low-latency VLN
• Batched action verification
• Two-stage alignment scheme
• Entropy-based adaptive generator