RFMSR: Residual Flow Matching for Image Super-Resolution

2026-07-14

AI Summary

This work addresses the limitations of existing image super-resolution methods in flow matching, which often neglect structural information in low-quality inputs and compromise multi-step inference capability when accelerating to a single step. The authors propose a residual flow matching framework that centers the source distribution in the latent space of the low-quality image, effectively shortening the transport trajectory while preserving structural priors. A two-stage training strategy is introduced, jointly optimizing conditional flow matching, residual modeling, end-to-end single-step supervision, and full-time-step velocity field loss. The resulting method achieves efficient single-step generation without sacrificing high-quality multi-step inference, attaining state-of-the-art or superior performance in both perceptual quality and fidelity.

Abstract

Image super-resolution (ISR) has witnessed remarkable progress with diffusion models and flow matching. The dominant text-to-image (T2I) based approaches leverage large-scale foundation models as generative priors, achieving impressive perceptual quality but at the cost of massive model sizes and prohibitive training expenses. Recent flow-matching-based vision-only approaches have made significant strides; however, they adopt standard flow formulations that transport from a pure Gaussian prior to the data distribution, discarding the rich structural information already present in the low-quality (LQ) input. Furthermore, existing single-step acceleration techniques often forfeit the model's multi-step inference capability. In this paper, we propose Residual Flow Matching for Image Super-Resolution (RFMSR), a vision-only framework that centers the source distribution at the LQ latent, reducing transport distance and preserving structural priors throughout the flow trajectory. We further introduce a two-phase training strategy: Phase I pretrains the velocity field via conditional flow matching, while Phase II applies end-to-end supervision to the single-step prediction while retaining the velocity loss across all timesteps, achieving high-quality single-step generation without sacrificing multi-step refinement. Extensive experiments demonstrate that RFMSR achieves comparable or even superior perceptual quality compared to state-of-the-art (SOTA) methods. The source code is available at https://github.com/Faze-Hsw/RFMSR.

Problem

• image super-resolution
• flow matching
• structural priors
• single-step acceleration
• multi-step inference

Innovation

• Residual Flow Matching
• Image Super-Resolution
• Vision-only Generation
• Two-phase Training
• Single-step Acceleration