T2MBench: A Benchmark for Out-of-Distribution Text-to-Motion Generation

2026-02-14

AI Summary

This work addresses the lack of systematic evaluation of text-to-motion generation models under out-of-distribution (OOD) complex textual conditions. To this end, the authors construct the first benchmark specifically designed for OOD scenarios, comprising 1,025 OOD textual descriptions, and introduce a unified multidimensional evaluation framework that integrates large language model–based assessment, multifactor motion quality metrics, and fine-grained accuracy evaluation. A comprehensive evaluation of 14 representative models using this framework reveals heterogeneous performance in semantic alignment, motion generalization, and physical plausibility, yet consistently highlights deficiencies in fine-grained accuracy. These findings provide clear guidance for future model development in text-to-motion generation.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Motion & Tracking

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Most existing evaluations of text-to-motion generation focus on in-distribution textual inputs and a limited set of evaluation criteria, which restricts their ability to systematically assess model generalization and motion generation capabilities under complex out-of-distribution (OOD) textual conditions. To address this limitation, we propose a benchmark specifically designed for OOD text-to-motion evaluation, which includes a comprehensive analysis of 14 representative baseline models and the two datasets derived from evaluation results. Specifically, we construct an OOD prompt dataset consisting of 1,025 textual descriptions. Based on this prompt dataset, we introduce a unified evaluation framework that integrates LLM-based Evaluation, Multi-factor Motion evaluation, and Fine-grained Accuracy Evaluation. Our experimental results reveal that while different baseline models demonstrate strengths in areas such as text-to-motion semantic alignment, motion generalizability, and physical quality, most models struggle to achieve strong performance with Fine-grained Accuracy Evaluation. These findings highlight the limitations of existing methods in OOD scenarios and offer practical guidance for the design and evaluation of future production-level text-to-motion models.

Problem

• out-of-distribution
• text-to-motion generation
• evaluation benchmark
• generalization
• motion generation

Innovation

• out-of-distribution
• text-to-motion
• benchmark
• fine-grained evaluation
• LLM-based evaluation