Robots as Tokens: Unified Diffusion Transformer for Coordinated Multi-Robot Trajectory Generation

2026-06-13

AI Summary

Existing multi-robot trajectory planning approaches often rely on sequential generation and post-processing, struggling to simultaneously ensure global coordination and safety constraints. This work proposes Roken, a novel framework that, for the first time, models multi-robot trajectories as spatiotemporal distributions and enables end-to-end parallel generation through tokenized representations and a diffusion-based Transformer. Roken integrates self-attention and cross-attention mechanisms and introduces a Bayesian-theorem-based multi-scale auxiliary task to enhance the learning efficiency of conditional distributions. Experiments demonstrate that Roken significantly outperforms baseline methods in complex, crowded scenarios, efficiently satisfying both individual safety and global connectivity constraints. Moreover, it exhibits strong generalization and scalability under varying team sizes, partial observability, and previously unseen environments.

Abstract

The success of generative models in language and visual generation has inspired extensive applications to generative robot planning. However, most existing works either focus on single-robot planning, or generate multi-robot trajectories in a sequential manner with iterative post-processing to resolve inter-robot conflicts. In this work, we investigate whether coordinated multi-robot trajectories, as a special spatiotemporal distribution, can be learned and generated with a generative model in a feed-forward manner. We propose Robots as Tokens (Roken), a unified diffusion transformer that directly generates multi-robot trajectories that satisfy both (individual) safety and (global) connectivity constraints. The core design of Roken is to represent each robot as a discrete token, allowing them to naturally interact with each other through self-attention, and cross-attend to map tokens for environment layouts. We further introduce several auxiliary tasks based on Bayes' theorem to provide multi-scale spatial-temporal supervision for efficient learning of the conditional distribution. In training, Roken absorbs diverse expert trajectories from different team sizes. During inference, Roken behaves as a versatile multi-robot planner that can handle single-robot planning, coordinated multi-robot trajectory generation, and conditional trajectory generation by fixing some robot tokens as conditions. Experiments in diverse cluttered environments show that Roken can generate coordinated multi-robot trajectories to perform connectivity-constrained goal navigation tasks with high success rates, outperforming the baseline method used to generate the training dataset. Roken also demonstrates good scalability after training with mixed team sizes, and shows generalization to unseen or partially observed environments, verifying its potential to learn from diverse data and perform versatile tasks.

Problem

• multi-robot trajectory generation
• coordinated planning
• spatiotemporal distribution
• connectivity constraints
• generative robot planning

Innovation

• diffusion transformer
• multi-robot trajectory generation
• robot tokens
• self-attention
• conditional generative planning