SkipSR: Faster Super Resolution with Token Skipping

2025-10-09

AI Summary

Diffusion-based video super-resolution (VSR) suffers from high computational overhead, hindering scalability to high-resolution and long-duration videos. To address this, we propose a content-aware spatial token skipping mechanism: leveraging low-resolution inputs, our method directly predicts spatially redundant regions and dynamically skips diffusion steps for those regions, applying super-resolution enhancement only where needed. This is the first work to introduce token skipping into VSR—requiring no additional training or architectural modifications to the underlying diffusion model—and remains fully compatible with both standard multi-step and single-step diffusion frameworks. Evaluated on 720p videos, our approach achieves up to 60% end-to-end latency reduction while preserving visual quality: PSNR and SSIM remain stable, and subjective assessments reveal no perceptible degradation. The method significantly improves inference efficiency and practical deployability without compromising fidelity.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Web Mining and Content Analysis: Content-based information diffusion
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Diffusion-based super-resolution (SR) is a key component in video generation and video restoration, but is slow and expensive, limiting scalability to higher resolutions and longer videos. Our key insight is that many regions in video are inherently low-detail and gain little from refinement, yet current methods process all pixels uniformly. To take advantage of this, we propose SkipSR, a simple framework for accelerating video SR by identifying low-detail regions directly from low-resolution input, then skipping computation on them entirely, only super-resolving the areas that require refinement. This simple yet effective strategy preserves perceptual quality in both standard and one-step diffusion SR models while significantly reducing computation. In standard SR benchmarks, our method achieves up to 60% faster end-to-end latency than prior models on 720p videos with no perceptible loss in quality. Video demos are available at https://rccchoudhury.github.io/skipsr/

Problem

• Accelerating slow diffusion-based video super-resolution
• Reducing computation by skipping low-detail regions
• Maintaining quality while achieving faster processing speeds

Innovation

• Skipping computation on low-detail video regions
• Identifying low-detail areas from low-resolution input
• Accelerating video super-resolution while preserving quality