Whole-Body Bilateral Teleoperation with Multi-Stage Object Parameter Estimation for Wheeled Humanoid Locomanipulation

2025-08-13

AI Summary

In whole-body bilateral teleoperation of wheeled humanoid robots, unknown object inertial parameters degrade dynamic synchronization, distort force feedback, and reduce operational tracking accuracy. To address this, we propose a multi-stage online inertial parameter estimation framework that integrates visual dimension measurement, large vision-language model (VLM)-based prior reasoning, hierarchical decoupled sampling, and multi-hypothesis validation—enabled by high-fidelity simulation and parallel hardware computation for millisecond-level parameter updates. The estimated parameters are embedded into whole-body dynamics compensation and balance control loops to enhance closed-loop teleoperation performance. Experiments on mobile manipulation tasks—such as lifting, transporting, and releasing payloads up to one-third the robot’s own weight—demonstrate significant improvements: 37% reduction in RMSE for tracking accuracy and enhanced interaction compliance. Results validate the method’s real-time capability, robustness, and physical plausibility.

Technology Category

• Intelligent Robots: State Estimation
• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper presents an object-aware whole-body bilateral teleoperation framework for wheeled humanoid loco-manipulation. This framework combines whole-body bilateral teleoperation with an online multi-stage object inertial parameter estimation module, which is the core technical contribution of this work. The multi-stage process sequentially integrates a vision-based object size estimator, an initial parameter guess generated by a large vision-language model (VLM), and a decoupled hierarchical sampling strategy. The visual size estimate and VLM prior offer a strong initial guess of the object's inertial parameters, significantly reducing the search space for sampling-based refinement and improving the overall estimation speed. A hierarchical strategy first estimates mass and center of mass, then infers inertia from object size to ensure physically feasible parameters, while a decoupled multi-hypothesis scheme enhances robustness to VLM prior errors. Our estimator operates in parallel with high-fidelity simulation and hardware, enabling real-time online updates. The estimated parameters are then used to update the wheeled humanoid's equilibrium point, allowing the operator to focus more on locomotion and manipulation. This integration improves the haptic force feedback for dynamic synchronization, enabling more dynamic whole-body teleoperation. By compensating for object dynamics using the estimated parameters, the framework also improves manipulation tracking while preserving compliant behavior. We validate the system on a customized wheeled humanoid with a robotic gripper and human-machine interface, demonstrating real-time execution of lifting, delivering, and releasing tasks with a payload weighing approximately one-third of the robot's body weight.

Problem

• Estimating object inertial parameters for teleoperation
• Enhancing dynamic whole-body bilateral teleoperation
• Improving manipulation tracking with compliant behavior

Innovation

• Multi-stage object inertial parameter estimation
• Vision-language model prior for initial guess
• Decoupled hierarchical sampling strategy