HANDO: Hierarchical Autonomous Navigation and Dexterous Omni-loco-manipulation

2025-10-10

AI Summary

Legged robots struggle to simultaneously achieve autonomous exploration and dexterous physical interaction in unstructured environments. To address this, we propose a hierarchical loco-manipulation framework: a high-level, semantic-goal-conditioned autonomous exploration strategy for target localization and navigation; and a low-level, unified whole-body torque control policy that coordinates leg and manipulator motions for fine-grained force interaction. The framework decouples high-level semantic decision-making from low-level motion execution, enabling, for the first time on embodied legged robots, unified real-time closed-loop modeling of semantic navigation, dynamic target search, and multi-joint coordinated manipulation. Experiments demonstrate the navigation module’s ability to accurately localize specified semantic targets in dynamic scenes, establishing a systematic foundation for end-to-end mobile manipulation.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Natural Language Processing: Safety and Robustness

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Seamless loco-manipulation in unstructured environments requires robots to leverage autonomous exploration alongside whole-body control for physical interaction. In this work, we introduce HANDO (Hierarchical Autonomous Navigation and Dexterous Omni-loco-manipulation), a two-layer framework designed for legged robots equipped with manipulators to perform human-centered mobile manipulation tasks. The first layer utilizes a goal-conditioned autonomous exploration policy to guide the robot to semantically specified targets, such as a black office chair in a dynamic environment. The second layer employs a unified whole-body loco-manipulation policy to coordinate the arm and legs for precise interaction tasks-for example, handing a drink to a person seated on the chair. We have conducted an initial deployment of the navigation module, and will continue to pursue finer-grained deployment of whole-body loco-manipulation.

Problem

• Enables legged robots to perform autonomous exploration in dynamic environments
• Coordinates whole-body control for precise loco-manipulation tasks
• Facilitates human-centered mobile manipulation using hierarchical framework

Innovation

• Two-layer framework for legged robots with manipulators
• Goal-conditioned autonomous exploration for target navigation
• Unified whole-body policy coordinating arm and leg movements