Problem
Research questions and friction points this paper is trying to address.
Zero-shot Temporal Action Localization
local correlations
feature representation
untrimmed videos
temporal action detection
ConTrans: Learning Text-enhanced Local-global Temporal Representations for Zero-shot Temporal Action Localization
🤖 AI Summary
This work addresses the limitations in zero-shot temporal action localization, particularly the neglect of local relative frame offsets and insufficient feature representation capacity. To overcome these issues, the authors propose ConTrans, a novel architecture that, for the first time, jointly models local relative offsets and global semantics. ConTrans integrates convolutional inductive bias with Transformer self-attention, employing a multi-scale encoder to simultaneously capture fine-grained local dependencies and long-range global context. Furthermore, it introduces text-enhanced learning of local-global temporal representations. Extensive experiments on ActivityNet-1.3 and THUMOS14 demonstrate that ConTrans significantly outperforms existing methods, establishing new state-of-the-art performance in zero-shot temporal action localization.
📝 Abstract
Zero-shot Temporal Action Localization (ZS-TAL) aims to detect and locate previously unseen actions in untrimmed videos. However, existing approaches primarily focus on modeling long-range contextual information, often neglecting the critical relative-offset-based local correlations between video frames. Furthermore, their performance is hindered by limited feature representation capabilities due to the shallow nature of their network architectures. In this paper, we address these limitations by introducing a novel local-global multi-scale feature representation module. We propose a novel multi-scale encoder architecture, termed ConTrans, that integrates convolutional (Conv) inductive biases with transformer Self-attention to jointly capture fine-grained local dependencies and long-range global context, leading to more comprehensive feature representations than existing methods. Experimental evaluations on the ActivityNet-1.3 and THUMOS14 datasets demonstrate that ConTrans significantly outperforms existing methods, establishing a new benchmark for ZS-TAL.
Innovation
Methods, ideas, or system contributions that make the work stand out.
ConTrans
zero-shot temporal action localization
local-global representation
convolutional inductive bias
multi-scale encoder
🔎 Similar Papers
No similar papers found.
