Faithful Extreme Image Rescaling with Learnable Reversible Transformation and Semantic Priors

2026-05-01

AI Summary

This work proposes FaithEIR, a diffusion-based framework designed to address the challenges of extreme image super-resolution (≥16×), where semantic structures are prone to distortion and generated details often lack realism. FaithEIR introduces a learnable invertible transformation in the latent space, inspired by singular value decomposition, to enable precise scaling control. To enhance semantic consistency and texture fidelity, the method incorporates an adaptive high-frequency detail prior dictionary and a lightweight pixel-wise semantic embedder. Furthermore, a semantic conditioning guidance mechanism is employed to fine-tune a pre-trained diffusion model. Extensive experiments demonstrate that FaithEIR significantly outperforms current state-of-the-art methods in both reconstruction fidelity and perceptual quality.

Abstract

Most recent extreme rescaling methods struggle to preserve semantically consistent structures and produce realistic details, due to the severely ill-posed nature of low- to high-resolution mapping under scaling factors of $16\times$ or higher. To alleviate the above problems, we propose FaithEIR, a diffusion-based framework for extreme image rescaling. Inspired by singular value decomposition, we develop learnable reversible transformation that enables invertible downscaling and upscaling in the latent space. To compensate for information loss due to quantization, we propose an adaptive detail prior, a high-frequency dictionary that captures the empirical average of commonly occurring structures in the training data. Finally, we design a lightweight pixel semantic embedder to provide semantic conditioning for the pretrained diffusion model. We present extensive experimental results demonstrating that our FaithEIR consistently outperforms state-of-the-art methods, achieving superior reconstruction fidelity and perceptual quality. Our code, model weights, and detailed results are released at https://github.com/cshw2021/FaithEIR.

Problem

• extreme image rescaling
• semantic consistency
• realistic details
• ill-posed mapping
• high-resolution reconstruction

Innovation

• learnable reversible transformation
• semantic priors
• diffusion-based rescaling
• adaptive detail prior
• extreme image rescaling