Auto-Encoded Supervision for Perceptual Image Super-Resolution

2024-11-28

arXiv.org

AI Summary

In GAN-based super-resolution, pixel-wise ℓp loss (Lpix) induces texture blurring and hinders simultaneous optimization of fidelity (PSNR) and perceptual quality (LPIPS). To address this, we propose AESOP—a novel supervision paradigm that defines reconstruction error within the decoding space of a pre-trained autoencoder (AE). AESOP is the first method to relocate Lpix into the AE’s latent space, effectively disentangling its blur-inducing components without altering network architecture or introducing auxiliary modules. The resulting AESOP loss function enables joint optimization of PSNR and LPIPS. Extensive experiments on multiple benchmark datasets demonstrate that AESOP significantly improves the PSNR–LPIPS trade-off. It is plug-and-play compatible with mainstream SR models, yielding high-resolution images with sharper textures and more natural visual appearance.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

This work tackles the fidelity objective in the perceptual super-resolution~(SR). Specifically, we address the shortcomings of pixel-level $L_ ext{p}$ loss ($mathcal{L}_ ext{pix}$) in the GAN-based SR framework. Since $L_ ext{pix}$ is known to have a trade-off relationship against perceptual quality, prior methods often multiply a small scale factor or utilize low-pass filters. However, this work shows that these circumventions fail to address the fundamental factor that induces blurring. Accordingly, we focus on two points: 1) precisely discriminating the subcomponent of $L_ ext{pix}$ that contributes to blurring, and 2) only guiding based on the factor that is free from this trade-off relationship. We show that they can be achieved in a surprisingly simple manner, with an Auto-Encoder (AE) pretrained with $L_ ext{pix}$. Accordingly, we propose the Auto-Encoded Supervision for Optimal Penalization loss ($L_ ext{AESOP}$), a novel loss function that measures distance in the AE space, instead of the raw pixel space. Note that the AE space indicates the space after the decoder, not the bottleneck. By simply substituting $L_ ext{pix}$ with $L_ ext{AESOP}$, we can provide effective reconstruction guidance without compromising perceptual quality. Designed for simplicity, our method enables easy integration into existing SR frameworks. Experimental results verify that AESOP can lead to favorable results in the perceptual SR task.

Problem

• Addressing fidelity issues in perceptual image super-resolution
• Overcoming blurring trade-offs in GAN-based SR frameworks
• Proposing AE-based loss to maintain perceptual quality

Innovation

• Uses Auto-Encoder for supervision in SR
• Proposes AESOP loss in AE space
• Maintains perceptual quality without blurring