IAE-VTG: Interaction-Aligned Action-Entity Video Temporal Grounding

2026-09-09

AI Summary

该研究解决了视频时间定位中动作与实体未同时出现的问题，通过细粒度解缠交互模块和交互敏感分配方法提升定位准确性。

Abstract

Video Temporal Grounding (VTG) localizes the video segment that matches a natural-language query. Many queries describe an action performed by a particular entity. Existing methods often encode the query as a whole or use general video-text interactions, without explicitly checking whether the action and entity occur together. They may therefore select a segment that contains both concepts but not the event described by the query. We propose Interaction Aligned Action-Entity Video Temporal Grounding (IAE-VTG), which models this rela?tionship at both the representation and training assignment levels. First, the Fine-grained Disentangled Interaction Module (FDIM) separates action and entity related query information and aligns it with complementary motion and appearance features. It then combines token-level interactions to build representations that capture the relationship between the action and entity. Second, Interaction-Sensitive Assignment (ISA) adds this interaction evidence to bipartite matching, so training targets are selected using both temporal overlap and semantic compatibility. This reduces supervision from temporally plausible but semantically incorrect proposals. Experiments on QVHighlights, Charades?STA, and TACoS show that IAE-VTG consistently improves strong baselines and achieves competitive or state-of-the-art performance on standard grounding metrics. Additional analyses show that the method is especially effective when similar actions or entities appear at multiple times and produces more reliable assignments for complex events.

Problem

• Video Temporal Grounding
• Natural-Language Query
• Action-Entity Relationship

Innovation

• Interaction Aligned Action-Entity
• Fine-grained Disentangled Interaction Module (FDIM)
• Interaction-Sensitive Assignment (ISA)