Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation

2025-02-27

AI Summary

This work addresses the problem of real-time, two-person interactive motion generation. We propose the first online dual-agent framework supporting asynchronous, autonomous responses. Methodologically, we integrate diffusion modeling with autoregressive architecture, designing two independent “brains”—one per agent—that streamingly react to the partner’s motion history. A dedicated diffusion head mitigates error accumulation in long sequences and enables sparse conditional control, ensuring compatibility with low-latency applications such as VR. Our core contributions are: (1) the first formalization of decentralized, asynchronous, real-time reaction mechanisms between two agents; and (2) significantly improved stability and controllability for extended motion sequences. Evaluated on highly dynamic boxing tasks, our method surpasses existing state-of-the-art approaches in motion quality, temporal coherence, and real-time performance—achieving frame rates sufficient for online interactive applications.

Technology Category

• Multiagent Systems: Adversarial Agents
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

This paper addresses the task of generating two-character online interactions. Previously, two main settings existed for two-character interaction generation: (1) generating one's motions based on the counterpart's complete motion sequence, and (2) jointly generating two-character motions based on specific conditions. We argue that these settings fail to model the process of real-life two-character interactions, where humans will react to their counterparts in real time and act as independent individuals. In contrast, we propose an online reaction policy, called Ready-to-React, to generate the next character pose based on past observed motions. Each character has its own reaction policy as its"brain", enabling them to interact like real humans in a streaming manner. Our policy is implemented by incorporating a diffusion head into an auto-regressive model, which can dynamically respond to the counterpart's motions while effectively mitigating the error accumulation throughout the generation process. We conduct comprehensive experiments using the challenging boxing task. Experimental results demonstrate that our method outperforms existing baselines and can generate extended motion sequences. Additionally, we show that our approach can be controlled by sparse signals, making it well-suited for VR and other online interactive environments.

Problem

• Generating real-time two-character interactions
• Reacting dynamically to observed motions
• Mitigating error accumulation in motion generation

Innovation

• Online reaction policy
• Diffusion head integration
• Character-specific reaction brains