DiffST: Spatiotemporal-Aware Diffusion for Real-World Space-Time Video Super-Resolution

2026-05-13

AI Summary

Existing diffusion-based spatiotemporal video super-resolution methods suffer from low inference efficiency and inadequate spatiotemporal modeling, hindering practical deployment. This work proposes an efficient spatiotemporal-aware video diffusion framework that, for the first time, integrates a one-step sampling mechanism into real-world video super-resolution. To enhance global spatiotemporal consistency, the framework introduces a Cross-Frame Context Aggregation (CFCA) module and a Video Representation Guidance (VRG) module. Employing an end-to-end full-video processing architecture, the method achieves high-quality reconstruction while accelerating inference by approximately 17× compared to prior diffusion-based approaches, attaining state-of-the-art performance in real-world scenarios.

Abstract

Diffusion-based models have shown strong performance in video super-resolution (VSR) and video frame interpolation (VFI). However, their role in the coupled space-time video super-resolution (STVSR) setting remains limited. Existing diffusion-based STVSR approaches suffer from two issues: (1) low inference efficiency and (2) insufficient utilization of spatiotemporal information. These limitations impede deployment. To address these issues, we introduce DiffST, an efficient spatiotemporal-aware video diffusion framework for real-world STVSR. To improve efficiency, we adapt a pre-trained diffusion model for one-step sampling and process the entire video directly rather than operating on individual frames. Furthermore, to enhance spatiotemporal information utilization, we introduce cross-frame context aggregation (CFCA) and video representation guidance (VRG). The CFCA module aggregates information across multiple keyframes to produce intermediate frames. The VRG module extracts video-level global features to guide the diffusion process. Extensive experiments show that DiffST obtains leading results on real-world STVSR tasks. It also maintains high inference efficiency, running about 17$\times$ faster than previous diffusion-based STVSR methods. Code is available at: https://github.com/zhengchen1999/DiffST.

Problem

• space-time video super-resolution
• diffusion models
• spatiotemporal information
• inference efficiency
• real-world video restoration

Innovation

• spatiotemporal-aware diffusion
• space-time video super-resolution
• one-step sampling
• cross-frame context aggregation
• video representation guidance