M3Bench: Benchmarking Whole-body Motion Generation for Mobile Manipulation in 3D Scenes

2024-10-09

arXiv.org

AI Summary

This paper addresses the challenge of base-arm coordinated motion planning for mobile manipulation. We introduce M3Bench, the first full-body motion generation benchmark for object rearrangement, comprising 119 diverse 3D scenes and 30,000 tasks. Methodologically, we propose M3BenchMaker—a fully automated data synthesis tool that generates high-quality expert trajectories from minimal inputs (scene geometry and robot kinematic parameters)—and employ physics-based, high-fidelity simulation for task-driven trajectory generation and structured constraint modeling. Our key contributions are: (1) the first mobile manipulation benchmark enabling multi-axis generalization evaluation; (2) a scalable, low-barrier data synthesis paradigm; and (3) empirical identification of systematic deficiencies in existing models—particularly in base-arm coordination and adherence to environmental and task-specific constraints—thereby establishing a standardized evaluation platform and foundational data infrastructure for next-generation mobile manipulation models.

Technology Category

• Intelligent Robots: Manipulation
• Computer Vision: Motion & Tracking
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

We propose M3Bench, a new benchmark for whole-body motion generation in mobile manipulation tasks. Given a 3D scene context, M3Bench requires an embodied agent to reason about its configuration, environmental constraints, and task objectives to generate coordinated whole-body motion trajectories for object rearrangement. M3Bench features 30,000 object rearrangement tasks across 119 diverse scenes, providing expert demonstrations generated by our newly developed M3BenchMaker, an automatic data generation tool that produces whole-body motion trajectories from high-level task instructions using only basic scene and robot information. Our benchmark includes various task splits to evaluate generalization across different dimensions and leverages realistic physics simulation for trajectory assessment. Extensive evaluation analysis reveals that state-of-the-art models struggle with coordinating base-arm motion while adhering to environmental and task-specific constraints, underscoring the need for new models to bridge this gap. By releasing M3Bench and M3BenchMaker we aim to advance robotics research toward more adaptive and capable mobile manipulation in diverse, real-world environments.

Problem

• Benchmarking whole-body motion generation for mobile manipulation in 3D scenes
• Evaluating coordination of base-arm motion under environmental constraints
• Advancing robotics research for adaptive mobile manipulation in diverse environments

Innovation

• M3Bench benchmarks whole-body motion generation
• M3BenchMaker automates expert trajectory generation
• Evaluates models with physics simulation splits