A Self-Supervised Learning Framework for Video Encoding Complexity Clustering

2026-06-27

AI Summary

This work addresses the challenge of determining optimal encoding parameters in adaptive video streaming due to substantial content variations across videos. To this end, the authors propose the Compression Echo Contrastive Learning (CECL) framework, which leverages “compression echoes”—artifacts inherently generated during video compression—as a self-supervised signal. By integrating contrastive learning with deep representation learning, CECL enables effective clustering of videos according to their encoding complexity without requiring any manual annotations. Experimental results demonstrate that CECL outperforms existing visual encoders on downstream clustering tasks and significantly surpasses conventional fixed bitrate ladder approaches, achieving both bitrate savings and improved visual quality.

Abstract

Adaptive video streaming is a widely used technique for delivering video content over the internet. One of the key challenges is determining the optimal encoding settings for each video, which can vary significantly based on its content and characteristics. In this paper, we propose Compression Echo Contrastive Learning (CECL), a novel self-supervised learning framework for clustering videos based on their encoding complexity. Our method leverages the response of a video to compression - the Compression Echo - as a supervisory signal, allowing the model to capture underlying encoding characteristics during pretraining. We conduct extensive experiments to demonstrate the effectiveness of our learned representations for the downstream task of clustering videos by their encoding complexity. Our results show that CECL improves upon existing state-of-the-art visual encoders and delivers strong bitrate and quality savings against the fixed bitrate ladder.

Problem

• video encoding complexity
• adaptive video streaming
• compression
• clustering
• self-supervised learning

Innovation

• Self-supervised learning
• Video encoding complexity
• Compression Echo
• Contrastive learning
• Adaptive video streaming