🤖 AI Summary
To address the instability of adversarial training and bias inherent in handcrafted reward functions in unsupervised video summarization, this paper proposes a two-stage decoupled reinforcement learning framework. In the first stage, a self-supervised pre-trained video reconstruction model generates frame-level reconstruction fidelity scores, serving as a learnable, data-driven reward signal. In the second stage, Proximal Policy Optimization (PPO) optimizes an importance-weighted summarization policy, with end-to-end differentiable approximation enabling efficient training. Crucially, the method eliminates heuristic reward design and adversarial training, being the first to directly model reconstruction quality as the RL reward. Evaluated on TVSum and SumMe, it achieves F-scores of 62.3 and 54.5, respectively—outperforming prior work—while accelerating inference by 300× over the state-of-the-art. Moreover, the generated summary distributions exhibit significantly improved alignment with human annotations.
📝 Abstract
This paper presents a novel approach for unsupervised video summarization using reinforcement learning. It aims to address the existing limitations of current unsupervised methods, including unstable training of adversarial generator-discriminator architectures and reliance on hand-crafted reward functions for quality evaluation. The proposed method is based on the concept that a concise and informative summary should result in a reconstructed video that closely resembles the original. The summarizer model assigns an importance score to each frame and generates a video summary. In the proposed scheme, reinforcement learning, coupled with a unique reward generation pipeline, is employed to train the summarizer model. The reward generation pipeline trains the summarizer to create summaries that lead to improved reconstructions. It comprises a generator model capable of reconstructing masked frames from a partially masked video, along with a reward mechanism that compares the reconstructed video from the summary against the original. The video generator is trained in a self-supervised manner to reconstruct randomly masked frames, enhancing its ability to generate accurate summaries. This training pipeline results in a summarizer model that better mimics human-generated video summaries compared to methods relying on hand-crafted rewards. The training process consists of two stable and isolated training steps, unlike adversarial architectures. Experimental results demonstrate promising performance, with F-scores of 62.3 and 54.5 on TVSum and SumMe datasets, respectively. Additionally, the inference stage is 300 times faster than our previously reported state-of-the-art method.