Training-Free Temporal Abstraction for General Video Understanding

2026-08-28

AI Summary

本文提出STITCH，一种无需训练的方法，通过预训练的视频-文本模型将视频分割成有意义的时间块，以支持多种视频理解任务。

Abstract

Videos are expensive to analyze frame by frame, yet many video understanding tasks depend on knowing where relevant moments occur. A system may need to find when an action changes, locate the segment described by a sentence, or choose a few frames for a vision-language model. Existing methods often solve these problems separately, using task-specific training data or specialized architectures. We study whether a pretrained video-text model can provide enough temporal structure to support several of these tasks at once. We present STITCH, a training-free method that divides a video into semantically meaningful temporal chunks. STITCH embeds short video windows with a frozen video-text backbone and detects changes in the resulting embedding sequence. These chunks are computed once per video and reused across tasks. We evaluate STITCH on generic event boundary detection, language-based moment retrieval, and frame selection for long-video VLM reasoning. Across all three settings, STITCH remains competitive with more specialized methods while requiring no task-specific training, with especially clear gains when only a small number of frames or tokens can be processed. These results suggest that reusable temporal abstraction is a promising direction for general video understanding, allowing dense video streams to be converted once into semantic units that can be localized, retrieved, sampled, or reasoned over by downstream systems.

Problem

• video understanding
• temporal abstraction
• pretrained model
• task-specific training
• semantic chunks

Innovation

• Training-Free
• Temporal Abstraction
• Pretrained Video-Text Model
• Semantic Chunks
• General Video Understanding