Embodied Navigation Foundation Model

2025-09-15

AI Summary

Current large vision-language models (VLMs) exhibit limited generalization in embodied navigation, struggling to transfer across diverse robotic morphologies and tasks. To address this, we propose NavFoM—a navigation foundation model trained on 8 million cross-platform navigation trajectories spanning quadrupeds, drones, wheeled robots, and autonomous vehicles. NavFoM employs a unified architecture augmented with learnable identifier tokens to accommodate heterogeneous sensor configurations and temporal horizons, and introduces a dynamic observation token sampling strategy for efficient inference under constrained compute budgets. Leveraging multimodal unified modeling, cross-domain data fusion, and large-scale pretraining, NavFoM achieves zero-shot transfer across morphologies and tasks—including vision-language navigation, goal-oriented search, target tracking, and autonomous driving—for the first time. It attains state-of-the-art or near-state-of-the-art performance across multiple benchmarks, demonstrating strong generalization and practical deployability in real-world settings.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Navigation is a fundamental capability in embodied AI, representing the intelligence required to perceive and interact within physical environments following language instructions. Despite significant progress in large Vision-Language Models (VLMs), which exhibit remarkable zero-shot performance on general vision-language tasks, their generalization ability in embodied navigation remains largely confined to narrow task settings and embodiment-specific architectures. In this work, we introduce a cross-embodiment and cross-task Navigation Foundation Model (NavFoM), trained on eight million navigation samples that encompass quadrupeds, drones, wheeled robots, and vehicles, and spanning diverse tasks such as vision-and-language navigation, object searching, target tracking, and autonomous driving. NavFoM employs a unified architecture that processes multimodal navigation inputs from varying camera configurations and navigation horizons. To accommodate diverse camera setups and temporal horizons, NavFoM incorporates identifier tokens that embed camera view information of embodiments and the temporal context of tasks. Furthermore, to meet the demands of real-world deployment, NavFoM controls all observation tokens using a dynamically adjusted sampling strategy under a limited token length budget. Extensive evaluations on public benchmarks demonstrate that our model achieves state-of-the-art or highly competitive performance across multiple navigation tasks and embodiments without requiring task-specific fine-tuning. Additional real-world experiments further confirm the strong generalization capability and practical applicability of our approach.

Problem

• Generalizing navigation across diverse robot embodiments and tasks
• Overcoming limitations of vision-language models in embodied navigation
• Unifying multimodal inputs for cross-embodiment navigation foundation model

Innovation

• Unified architecture for multimodal navigation inputs
• Identifier tokens for camera views and temporal context
• Dynamic token sampling under limited length budget