Collaborative Multi-Robot Non-Prehensile Manipulation via Flow-Matching Co-Generation

2025-11-14

AI Summary

Coordinating multiple robots to rearrange numerous objects in cluttered environments requires joint optimization of contact point selection, non-prehensile manipulation trajectories, and large-scale anonymous navigation—posing significant challenges in scalability, physical feasibility, and autonomy. Method: We propose a generative co-design framework that, from visual input alone, end-to-end generates both contact configurations and manipulation trajectories for the first time. Leveraging flow matching-based generative modeling, it ensures high diversity and physical plausibility. The framework integrates scalable anonymous multi-robot path planning, unifying motion planning and task assignment without requiring object models or privileged information. Contribution/Results: The approach enables long-horizon, unsupervised collaborative rearrangement. Extensive simulations demonstrate substantial improvements over state-of-the-art baselines in success rate, efficiency, and robustness—establishing a new paradigm for large-scale multi-robot manipulation in complex, unstructured environments.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Coordinating a team of robots to reposition multiple objects in cluttered environments requires reasoning jointly about where robots should establish contact, how to manipulate objects once contact is made, and how to navigate safely and efficiently at scale. Prior approaches typically fall into two extremes -- either learning the entire task or relying on privileged information and hand-designed planners -- both of which struggle to handle diverse objects in long-horizon tasks. To address these challenges, we present a unified framework for collaborative multi-robot, multi-object non-prehensile manipulation that integrates flow-matching co-generation with anonymous multi-robot motion planning. Within this framework, a generative model co-generates contact formations and manipulation trajectories from visual observations, while a novel motion planner conveys robots at scale. Crucially, the same planner also supports coordination at the object level, assigning manipulated objects to larger target structures and thereby unifying robot- and object-level reasoning within a single algorithmic framework. Experiments in challenging simulated environments demonstrate that our approach outperforms baselines in both motion planning and manipulation tasks, highlighting the benefits of generative co-design and integrated planning for scaling collaborative manipulation to complex multi-agent, multi-object settings. Visit gco-paper.github.io for code and demonstrations.

Problem

• Coordinating robot teams to reposition multiple objects in cluttered environments
• Integrating contact formation and trajectory generation from visual observations
• Unifying robot-level and object-level reasoning for collaborative manipulation

Innovation

• Flow-matching co-generation for contact and trajectory planning
• Anonymous multi-robot motion planning for scalable coordination
• Integrated robot-object reasoning within unified algorithmic framework