Burst Image Super-Resolution with Mamba

2025-03-25

AI Summary

To address the quadratic computational complexity and sub-pixel alignment challenges of Transformer-based architectures in burst multi-frame super-resolution (SR), this paper proposes a Mamba-driven dual-branch network. The spatial branch reconstructs the key frame, while the temporal branch leverages optical flow-guided state sequence modeling to extract sub-pixel motion priors. We introduce two key innovations: (i) an optical flow-guided serialized state update mechanism, and (ii) a wavelet-reparameterized state space model—enabling high-frequency information prioritization and precise sub-pixel alignment while retaining linear-time complexity. Our method achieves state-of-the-art performance on three major benchmarks—SyntheticSR, RealBSR-RGB, and RealBSR-RAW. Notably, it is the first work to successfully integrate efficient state space models into multi-frame SR, combining global spatiotemporal modeling capability with strong potential for real-time inference.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs

Abstract

Burst image super-resolution (BISR) aims to enhance the resolution of a keyframe by leveraging information from multiple low-resolution images captured in quick succession. In the deep learning era, BISR methods have evolved from fully convolutional networks to transformer-based architectures, which, despite their effectiveness, suffer from the quadratic complexity of self-attention. We see Mamba as the next natural step in the evolution of this field, offering a comparable global receptive field and selective information routing with only linear time complexity. In this work, we introduce BurstMamba, a Mamba-based architecture for BISR. Our approach decouples the task into two specialized branches: a spatial module for keyframe super-resolution and a temporal module for subpixel prior extraction, striking a balance between computational efficiency and burst information integration. To further enhance burst processing with Mamba, we propose two novel strategies: (i) optical flow-based serialization, which aligns burst sequences only during state updates to preserve subpixel details, and (ii) a wavelet-based reparameterization of the state-space update rules, prioritizing high-frequency features for improved burst-to-keyframe information passing. Our framework achieves SOTA performance on public benchmarks of SyntheticSR, RealBSR-RGB, and RealBSR-RAW.

Problem

• Enhancing resolution of keyframes using burst low-res images
• Reducing quadratic complexity in transformer-based BISR methods
• Balancing computational efficiency with burst information integration

Innovation

• Mamba-based architecture for burst super-resolution
• Optical flow-based serialization for alignment
• Wavelet-based reparameterization for high-frequency features