RTNav: Towards Real-Time Zero-Shot Object Navigation

2026-08-26

AI Summary

本文针对实时零样本物体导航中模型推理延迟问题，提出RTNav架构，考虑异步环境和计算限制，提升导航成功率和效率。

Abstract

Navigation in unknown environments to find unforeseen objects has become increasingly feasible with capable vision and language foundation models. However, these models also introduce non-negligible inference latency, which becomes an important concern when agents must operate continuously in the real world. Most state-of-the-art methods are still developed in synchronous simulators, where the environment waits for the agent to act and inference time is effectively free. As a result, agents are often designed around the sequential execution of perception, reasoning, and action, with little regard for time constraints. Under real-time execution, where wall-clock time counts towards the task budget, the inefficiencies of these architectures become clear. We show that recent zero-shot object navigation methods suffer consistent performance degradation under such realistic timing conditions. Motivated by this observation, we propose RTNav, a simple but effective architecture that treats inference latency, asynchronous environment stepping, and bounded compute as explicit design considerations. Evaluated on real-time variants of HM3D-v1, HM3D-v2, and HM3D-OVON, RTNav improves the success rate by up to 11% and the Success weighted by Completion Time by up to 5.1 points over prior work.

Problem

• Real-Time Navigation
• Inference Latency
• Zero-Shot Object Navigation
• Asynchronous Environment

Innovation

• Real-Time Navigation
• Zero-Shot Learning
• Inference Latency
• Asynchronous Environment Stepping
• Bounded Compute