UniLM-Nav: A Unified Framework for Zero-Shot Last-Mile Navigation

2026-07-07

AI Summary

This work addresses the challenge of achieving precise, actionable robot poses in open-vocabulary “last-mile” navigation for mobile robots. To this end, it proposes a zero-shot approach grounded in a unified multimodal large language model (MLLM), decomposing the task into view selection, task-conditioned localization of actionable points, and geometry-aware base pose reasoning. The method operates without human annotations or task-specific training, enabling fine-grained spatially constrained navigation. Evaluated on the OVMM benchmark, it outperforms the current state-of-the-art method MoTo by 3.13 percentage points and demonstrates successful real-world deployment on the Unitree B2 quadrupedal robot platform.

Abstract

Mobile manipulation requires a robot to navigate to a target object or receptacle and then perform intended manipulation. However, reaching the vicinity of the target does not guarantee a manipulation-ready base pose, a problem known as last-mile navigation. Prior methods for last-mile navigation either rely on manual pose annotation or task-specific training, limiting their scalability to open-vocabulary settings with fine-grained spatial constraints. We propose UniLM-Nav, a unified framework for zero-shot open-vocabulary last-mile navigation. UniLM-Nav decomposes last-mile navigation into view selection, task-conditioned affordance grounding, and geometry-aware base-pose reasoning, all resolved with a shared multimodal large language model (MLLM) backend. Specifically, UniLM-Nav first selects a reference view that best captures the target object or receptacle from recently collected observations. It then grounds task-relevant affordance point in the selected view and lifts the result into the robot-centric coordinate frame. Finally, conditioned on the grounded affordance, task context, and robot geometry, it infers a manipulation-ready base pose for the robot. We evaluate UniLM-Nav on the OVMM benchmark, where it outperforms the previous state-of-the-art method, MoTo, by 3.13 percentage points. Analyses show that the components of our method are crucial to final performance, and that the choice of MLLM also has a substantial effect. We further deploy UniLM-Nav on a Unitree B2 quadruped robot with a 6-DoF Unitree Z1 manipulator, validating its applicability to real-world mobile manipulation tasks.

Problem

• last-mile navigation
• mobile manipulation
• open-vocabulary
• manipulation-ready pose
• zero-shot

Innovation

• zero-shot navigation
• open-vocabulary
• multimodal large language model
• affordance grounding
• last-mile navigation