Interpretable Motion-Attentive Maps: Spatio-Temporally Localizing Concepts in Video Diffusion Transformers

2026-03-03

AI Summary

This work addresses the lack of interpretable spatiotemporal grounding for motion-related textual concepts in existing video diffusion Transformers. To this end, the authors propose GramCol, a framework that, for the first time, generates frame-level saliency maps for arbitrary textual concepts without requiring gradient computation or parameter updates. By integrating a motion-aware feature selection algorithm, GramCol constructs Interpretable Motion Attention Maps (IMAPs), enabling precise spatiotemporal localization of both motion and non-motion concepts. This approach overcomes the limitations of prior methods that focus solely on static object saliency, achieving superior performance in motion localization and zero-shot video semantic segmentation. The resulting saliency maps exhibit enhanced clarity and stronger semantic interpretability compared to existing techniques.

Technology Category

• Computer Vision: Motion & Tracking
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Video Diffusion Transformers (DiTs) have been synthesizing high-quality video with high fidelity from given text descriptions involving motion. However, understanding how Video DiTs convert motion words into video remains insufficient. Furthermore, while prior studies on interpretable saliency maps primarily target objects, motion-related behavior in Video DiTs remains largely unexplored. In this paper, we investigate concrete motion features that specify when and which object moves for a given motion concept. First, to spatially localize, we introduce GramCol, which adaptively produces per-frame saliency maps for any text concept, including both motion and non-motion. Second, we propose a motion-feature selection algorithm to obtain an Interpretable Motion-Attentive Map (IMAP) that localizes motion spatially and temporally. Our method discovers concept saliency maps without the need for any gradient calculation or parameter update. Experimentally, our method shows outstanding localization capability on the motion localization task and zero-shot video semantic segmentation, providing interpretable and clearer saliency maps for both motion and non-motion concepts.

Problem

• Video Diffusion Transformers
• motion interpretation
• saliency maps
• spatio-temporal localization
• interpretable AI

Innovation

• Interpretable Motion-Attentive Map
• Video Diffusion Transformers
• motion localization
• saliency map
• GramCol