Iterative Shaping of Multi-Particle Aggregates based on Action Trees and VLM

2025-01-23

AI Summary

This work addresses the challenge of autonomous shaping control for multi-particle aggregates. We propose a dual-arm robotic coordination framework that achieves active particle aggregation and precise shape regulation through sequenced shaping and pushing actions. Our method innovatively integrates vision-language model (VLM)-driven action-tree planning with dynamic aggregate contour representation based on truncated Fourier series, establishing a closed-loop shape evolution control architecture. Furthermore, we design an adaptive centroid–clustering trajectory generation strategy to ensure spatial cohesion throughout aggregation. Real-world experiments demonstrate that our approach reduces aggregate shape error by 42% over multiple iterative shaping cycles, while maintaining system spatial cohesion above 91%. These results significantly enhance controllability and morphological fidelity in multi-particle systems.

Technology Category

• Intelligent Robots: Manipulation
• Multiagent Systems: Coordination and Collaboration
• Computer Vision: Vision for Robotics & Autonomous Driving

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

In this paper, we address the problem of manipulating multi-particle aggregates using a bimanual robotic system. Our approach enables the autonomous transport of dispersed particles through a series of shaping and pushing actions using robotically-controlled tools. Achieving this advanced manipulation capability presents two key challenges: high-level task planning and trajectory execution. For task planning, we leverage Vision Language Models (VLMs) to enable primitive actions such as tool affordance grasping and non-prehensile particle pushing. For trajectory execution, we represent the evolving particle aggregate's contour using truncated Fourier series, providing efficient parametrization of its closed shape. We adaptively compute trajectory waypoints based on group cohesion and the geometric centroid of the aggregate, accounting for its spatial distribution and collective motion. Through real-world experiments, we demonstrate the effectiveness of our methodology in actively shaping and manipulating multi-particle aggregates while maintaining high system cohesion.

Problem

• Dual-arm Robotics
• Particle Management
• Agglomeration Integrity

Innovation

• Visual Language Model
• Truncated Fourier Series
• Bimanual Robot Operation