SynAgent: Generalizable Cooperative Humanoid Manipulation via Solo-to-Cooperative Agent Synergy

2026-04-20

AI Summary

This work addresses the challenges of data scarcity, complex multi-agent coordination, and limited cross-object generalization in controllable collaborative humanoid manipulation by proposing the Solo-to-Cooperative Agent Synergy framework, which transfers single-agent human-object interaction skills to multi-agent cooperative settings. The approach integrates interaction-preserving retargeting, a cooperation guidance mechanism built upon single-agent pretraining, and trajectory-conditioned generation, leveraging Interact Mesh (based on Delaunay tetrahedralization), decentralized multi-agent PPO, conditional VAEs, and multi-teacher distillation to significantly enhance control stability and generalization. Experiments demonstrate that the method outperforms existing baselines in both cooperative imitation and trajectory-conditioned control tasks and generalizes effectively across diverse object geometries.

Technology Category

• Multiagent Systems: Teamwork
• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Controllable cooperative humanoid manipulation is a fundamental yet challenging problem for embodied intelligence, due to severe data scarcity, complexities in multi-agent coordination, and limited generalization across objects. In this paper, we present SynAgent, a unified framework that enables scalable and physically plausible cooperative manipulation by leveraging Solo-to-Cooperative Agent Synergy to transfer skills from single-agent human-object interaction to multi-agent human-object-human scenarios. To maintain semantic integrity during motion transfer, we introduce an interaction-preserving retargeting method based on an Interact Mesh constructed via Delaunay tetrahedralization, which faithfully maintains spatial relationships among humans and objects. Building upon this refined data, we propose a single-agent pretraining and adaptation paradigm that bootstraps synergistic collaborative behaviors from abundant single-human data through decentralized training and multi-agent PPO. Finally, we develop a trajectory-conditioned generative policy using a conditional VAE, trained via multi-teacher distillation from motion imitation priors to achieve stable and controllable object-level trajectory execution. Extensive experiments demonstrate that SynAgent significantly outperforms existing baselines in both cooperative imitation and trajectory-conditioned control, while generalizing across diverse object geometries. Codes and data will be available after publication. Project Page: http://yw0208.github.io/synagent

Problem

• cooperative manipulation
• embodied intelligence
• multi-agent coordination
• generalization
• humanoid manipulation

Innovation

• Solo-to-Cooperative Agent Synergy
• Interaction-preserving Retargeting
• Decentralized Multi-agent PPO
• Trajectory-conditioned Generative Policy
• Interact Mesh