PersonaBooth: Personalized Text-to-Motion Generation

2025-03-10

AI Summary

This paper addresses text-driven personalized motion generation by introducing the novel task of “motion personalization”: generating stylistically consistent, personalized motion sequences from minimal motion samples annotated with human-specific attributes (Persona) and corresponding textual descriptions. To mitigate distribution shift caused by the lack of persona-specific supervision in pretraining and ensure cross-action-type feature consistency, we propose: (1) a dedicated Persona token to explicitly model individual traits; (2) multimodal fine-tuning jointly leveraging text and motion visual features; (3) contrastive learning to enhance intra-persona motion representation consistency; and (4) a context-aware fusion mechanism to integrate persona cues across heterogeneous motion sources. Experiments demonstrate that our method significantly outperforms existing motion style transfer approaches on the Motion Personalization task and establishes the first benchmark for this emerging domain.

Technology Category

• Computer Vision: Motion & Tracking
• Natural Language Processing: Generation
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This paper introduces Motion Personalization, a new task that generates personalized motions aligned with text descriptions using several basic motions containing Persona. To support this novel task, we introduce a new large-scale motion dataset called PerMo (PersonaMotion), which captures the unique personas of multiple actors. We also propose a multi-modal finetuning method of a pretrained motion diffusion model called PersonaBooth. PersonaBooth addresses two main challenges: i) A significant distribution gap between the persona-focused PerMo dataset and the pretraining datasets, which lack persona-specific data, and ii) the difficulty of capturing a consistent persona from the motions vary in content (action type). To tackle the dataset distribution gap, we introduce a persona token to accept new persona features and perform multi-modal adaptation for both text and visuals during finetuning. To capture a consistent persona, we incorporate a contrastive learning technique to enhance intra-cohesion among samples with the same persona. Furthermore, we introduce a context-aware fusion mechanism to maximize the integration of persona cues from multiple input motions. PersonaBooth outperforms state-of-the-art motion style transfer methods, establishing a new benchmark for motion personalization.

Problem

• Generates personalized motions aligned with text descriptions
• Addresses distribution gap between persona-focused and pretraining datasets
• Captures consistent persona from varying motion content

Innovation

• Introduces PersonaBooth for personalized motion generation
• Uses persona tokens for multi-modal adaptation
• Incorporates contrastive learning for consistent persona capture