MotionAtlas: Detailed Region Captioning for Motion-Centric Videos

2026-06-28

AI Summary

Existing video captioning methods struggle to accurately and finely characterize motion in specific regions of motion-dense videos, often hindered by visual distractions and motion entanglement. This work proposes a region-aware motion description paradigm that employs spatiotemporal masks to guide the model’s focus toward target regions and introduces, for the first time, a self-guided refinement mechanism to suppress fine-grained hallucinations. The study also constructs MotionAtlas-Bench—a new evaluation benchmark comprising 2,073 questions—and a dataset of 159,000 high-quality samples, complemented by a tailored training strategy. On general motion captioning benchmarks, the proposed MotionAtlas-4B model outperforms Qwen3-VL-4B by an average of 5.2 percentage points.

Abstract

We propose MotionAtlas, a system for detailed captioning of motion-centric videos, comprising (1) a dedicated human-annotated benchmark, (2) a scalable, high-quality pipeline to construct training samples, and (3) a family of powerful Video-MLLMs. Unlike conventional global motion captioning datasets, we focus on region-aware motion captioning: given a video and a spatiotemporal mask, the model generates precise descriptions of motion within the target region, thereby alleviating visual clutter and motion entanglement and enabling reliable, quantifiable evaluation. Concretely, we first build MotionAtlas-Bench, a comprehensive benchmark comprising 2,073 multiple-choice questions, meticulously annotated for a curated set of high-quality, motion-centric videos, to evaluate fine-grained motion understanding of the objects in question. Second, we design a rigorous and scalable data pipeline that leverages self-bootstrap refinement to suppress fine-grained hallucinations, yielding 159k high-quality motion captioning data. Third, we design a tailored training data composition strategy, which achieves consistent and substantial performance gains across diverse baseline Video-MLLMs, including Molmo2 and Qwen3-VL. For instance, MotionAtlas-4B surpasses Qwen3-VL-4B by an average of 5.2 percentage points across general motion benchmarks. The benchmark, dataset, and code have been released.

Problem

• region-aware motion captioning
• motion-centric videos
• fine-grained motion understanding
• visual clutter
• motion entanglement

Innovation

• region-aware motion captioning
• Video-MLLM
• self-bootstrap refinement
• motion-centric video understanding
• fine-grained hallucination suppression