AVSR-Diff: Scale-Agnostic Diffusion Priors for Temporally Consistent Arbitrary-Scale Video Super-Resolution

2026-07-01

AI Summary

Existing video super-resolution methods struggle to simultaneously preserve high-frequency details and temporal consistency across arbitrary scales, with diffusion models constrained to fixed scales and coordinate-based approaches suffering from oversmoothing at large magnifications. This work proposes a decoupled framework that separates scale-agnostic latent denoising from continuous coordinate rendering. To ensure temporal coherence, it introduces a Temporal Gated Feature Reuse (TGFR) mechanism, complemented by a continuous video VAE and a Scale-Aware Fourier Refinement (SAFR) module that dynamically adapts frequency-domain features to the target scale. The method is the first to effectively leverage diffusion priors for arbitrary-scale video super-resolution, consistently outperforming existing approaches across multiple scales—delivering richer high-frequency details, stronger temporal stability, and even surpassing state-of-the-art generative models specifically designed for fixed-scale tasks.

Abstract

Diffusion models have significantly advanced video super-resolution (VSR) but remain largely constrained to fixed upsampling scales. Conversely, while coordinate-based arbitrary-scale VSR methods offer scale flexibility, they inherently suffer from severe over-smoothing at large scaling factors. Integrating generative priors with continuous decoding is promising but currently hindered by severe temporal flickering caused by the stochasticity of diffusion sampling. To address this, we propose AVSR-Diff (Arbitrary-scale Video Super-Resolution with Diffusion), a novel decoupled framework that separates scale-agnostic latent denoising from continuous coordinate rendering, effectively avoiding computationally heavy resolution-specific sampling. Our approach introduces a Temporally-Gated Feature Recurrence (TGFR) module to extract strictly aligned, temporally consistent latent priors. Furthermore, we design a continuous video VAE decoder incorporating a Scale-Aware Fourier Refinement (SAFR) module to dynamically adapt frequency components to any target scale. Extensive experiments demonstrate that AVSR-Diff consistently preserves high-frequency details and strong temporal stability across various scales, surpassing state-of-the-art arbitrary-scale baselines. Remarkably, our framework outperforms recent fixed-scale generative models even on their native resolution.

Problem

• video super-resolution
• arbitrary-scale
• temporal consistency
• diffusion models
• over-smoothing

Innovation

• arbitrary-scale video super-resolution
• diffusion models
• temporal consistency
• scale-agnostic prior
• continuous coordinate rendering