AdaptSR: Low-Rank Adaptation for Efficient and Scalable Real-World Super-Resolution

2025-03-10

AI Summary

Real-world super-resolution (SR) faces challenges in recovering high-frequency details due to unknown, complex degradations; existing GAN-based methods suffer from training instability, while diffusion models incur prohibitive computational overhead. To address these issues, this paper proposes AdaptSR—the first lightweight, parameter-efficient adaptation framework for real-world SR that introduces Low-Rank Adaptation (LoRA). Without retraining the base model or introducing generative instability, AdaptSR builds upon a bicubic-interpolation-pretrained SR model and employs architecture-aware, hierarchical selective LoRA updates with weight merging, enabling zero-inference-overhead adaptation. Experiments on real-world SR benchmarks demonstrate a 4 dB PSNR gain and a 2% improvement in perceptual score—matching full-model fine-tuning performance—while training only 8% of the parameters and reducing training time by 92%. The method enables minute-level deployment on resource-constrained hardware.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Recovering high-frequency details and textures from low-resolution images remains a fundamental challenge in super-resolution (SR), especially when real-world degradations are complex and unknown. While GAN-based methods enhance realism, they suffer from training instability and introduce unnatural artifacts. Diffusion models, though promising, demand excessive computational resources, often requiring multiple GPU days, even for single-step variants. Rather than naively fine-tuning entire models or adopting unstable generative approaches, we introduce AdaptSR, a low-rank adaptation (LoRA) framework that efficiently repurposes bicubic-trained SR models for real-world tasks. AdaptSR leverages architecture-specific insights and selective layer updates to optimize real SR adaptation. By updating only lightweight LoRA layers while keeping the pretrained backbone intact, it captures domain-specific adjustments without adding inference cost, as the adapted layers merge seamlessly post-training. This efficient adaptation not only reduces memory and compute requirements but also makes real-world SR feasible on lightweight hardware. Our experiments demonstrate that AdaptSR outperforms GAN and diffusion-based SR methods by up to 4 dB in PSNR and 2% in perceptual scores on real SR benchmarks. More impressively, it matches or exceeds full model fine-tuning while training 92% fewer parameters, enabling rapid adaptation to real SR tasks within minutes.

Problem

• Efficiently recover high-frequency details from low-resolution images
• Address training instability and artifacts in GAN-based SR methods
• Reduce computational demands of diffusion models for real-world SR

Innovation

• Low-rank adaptation for efficient super-resolution
• Selective layer updates optimize real SR adaptation
• Lightweight LoRA layers reduce memory and compute