EHC-MM: Embodied Holistic Control for Mobile Manipulation

2024-09-13

arXiv.org

AI Summary

Addressing the challenge of tightly coupling base navigation, manipulator control, and visual perception in mobile manipulation robots, this paper proposes an embodied multimodal closed-loop control framework. Methodologically, it introduces three key innovations: (1) a differentiable embodied function σ(w) that dynamically balances prioritization between mobility and manipulation tasks; (2) a monitor-pose-guided servoing method (MPBS) ensuring robust end-to-end target tracking; and (3) a unified quadratic programming (QP)-based optimization model integrating kinematic constraints, visual feedback, and task-specific requirements. Experimental evaluation in real-world scenarios demonstrates significant performance improvements: task success rate increases to 95.6%, and average execution time decreases by 52.8% compared to conventional hierarchical control approaches. These results validate the framework’s effectiveness in enabling coordinated, responsive, and constraint-aware mobile manipulation.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Multi-modal Vision
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Mobile manipulation typically entails the base for mobility, the arm for accurate manipulation, and the camera for perception. It is necessary to follow the principle of Distant Mobility, Close Grasping(DMCG) in holistic control. We propose Embodied Holistic Control for Mobile Manipulation(EHC-MM) with the embodied function of sig(w): By formulating the DMCG principle as a Quadratic Programming (QP) problem, sig(w) dynamically balances the robot's emphasis between movement and manipulation with the consideration of the robot's state and environment. In addition, we propose the Monitor-Position-Based Servoing (MPBS) with sig(w), enabling the tracking of the target during the operation. This approach allows coordinated control between the robot's base, arm, and camera. Through extensive simulations and real-world experiments, our approach significantly improves both the success rate and efficiency of mobile manipulation tasks, achieving a 95.6% success rate in the real-world scenarios and a 52.8% increase in time efficiency.

Problem

• Dynamic balance between robot movement and manipulation
• Coordinated control of base, arm, and camera for efficiency
• Enhanced success rate and time efficiency in mobile manipulation

Innovation

• Embodied Holistic Control balances movement and manipulation.
• Quadratic Programming optimizes Distant Mobility, Close Grasping.
• Monitor-Position-Based Servoing enhances target tracking efficiency.