Multi-Task Visual Perception Network with LLM Conditioning for Autonomous Navigation

2026-09-13

AI Summary

为解决长期导航中里程计漂移和传感器误差累积问题，提出结合视觉感知与大语言模型的框架，基于局部视觉线索生成顺序动作计划，实现安全导航。

Abstract

Long-term navigation for service robots faces crit- ical challenges like the accumulation of odometry drift and sensor error, which progressively degrade 2D maps and renders traditional path planning algorithms (e.g., A*, RRT*, DiPPer, ViT-A*) ineffective over time. To address this, we propose a user-friendly, interactive framework that eliminates the reliance on globally consistent maps. Our approach integrates visual perception with Large Language Models (LLM) to interpret user commands via text or voice. Instead of relying on a drift- prone global map, the system generates a sequential action plan based on local visual cues and egocentric geometric instructions. These action plans are executed sequentially, allowing the robot to navigate known and unknown environments safely. By reset- ting localization relative to immediate targets, our framework effectively works with a minimum accumulation drift strategy, ensuring accurate, efficient, and collision-free navigation without the maintenance overhead of traditional mapping. Experiments on real-world and simulated data have shown significant improve- ments over other methods. Our source code is publicly accessible at https://github.com/PraveenSingh24/VL-Navigation.

Problem

• odometry drift
• sensor error
• autonomous navigation
• service robots
• path planning

Innovation

• Multi-Task Visual Perception
• LLM Conditioning
• Autonomous Navigation
• Localization Reset
• Sequential Action Plan