Diffusion-Based Imitation Learning for Social Pose Generation

2025-01-18

AI Summary

This work addresses the challenge of enabling socially appropriate nonverbal behavior imitation by AI agents acting as social guides in human–agent collaboration, specifically through learning human body poses. We propose a diffusion-based multi-agent pose sequence generation framework—the first to apply Diffusion Behavior Cloning (DBC) to social guidance tasks—and systematically evaluate the impact of raw versus preprocessed pose representations on conditional generation quality and efficiency. Experiments demonstrate that pose preprocessing significantly improves generation fidelity (reducing MPJPE by 18.7%) while maintaining inference efficiency, outperforming mainstream generative baselines in the accuracy–computational cost trade-off. Our core contribution is the establishment of the first diffusion-based pose generation paradigm tailored for social coordination tasks, alongside empirical validation of the critical role of data representation in modeling socially situated behaviors.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Diffusion Models for Vision
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Intelligent agents, such as robots and virtual agents, must understand the dynamics of complex social interactions to interact with humans. Effectively representing social dynamics is challenging because we require multi-modal, synchronized observations to understand a scene. We explore how using a single modality, the pose behavior, of multiple individuals in a social interaction can be used to generate nonverbal social cues for the facilitator of that interaction. The facilitator acts to make a social interaction proceed smoothly and is an essential role for intelligent agents to replicate in human-robot interactions. In this paper, we adapt an existing diffusion behavior cloning model to learn and replicate facilitator behaviors. Furthermore, we evaluate two representations of pose observations from a scene, one representation has pre-processing applied and one does not. The purpose of this paper is to introduce a new use for diffusion behavior cloning for pose generation in social interactions. The second is to understand the relationship between performance and computational load for generating social pose behavior using two different techniques for collecting scene observations. As such, we are essentially testing the effectiveness of two different types of conditioning for a diffusion model. We then evaluate the resulting generated behavior from each technique using quantitative measures such as mean per-joint position error (MPJPE), training time, and inference time. Additionally, we plot training and inference time against MPJPE to examine the trade-offs between efficiency and performance. Our results suggest that the further pre-processed data can successfully condition diffusion models to generate realistic social behavior, with reasonable trade-offs in accuracy and processing time.

Problem

• Humanoid Robots
• Social Behavior Learning
• Body Posture Imitation

Innovation

• Diffusion Behavior Cloning
• Social Skill Learning
• Preprocessed Information Observation