Learning from a Generative Oracle: Domain Adaptation for Restoration

2025-12-11

AI Summary

Real-world image degradation distributions are unknown and exhibit significant domain shift relative to pretraining data, severely limiting the zero-shot generalization capability of existing restoration models. To address this, we propose LEGO, a three-stage unpaired domain adaptation framework. First, a large-scale generative model (e.g., a diffusion model) is frozen as a generative oracle to synthesize high-fidelity pseudo-ground-truth images for unlabeled degraded inputs. Second, a pseudo-label refinement strategy is introduced to enhance label reliability. Third, hybrid supervised fine-tuning enables lightweight adaptation without architectural modification. LEGO requires no paired data, preserves robustness to the original distribution, and achieves zero-shot domain transfer. Extensive experiments on multiple real-world degradation benchmarks demonstrate substantial improvements over state-of-the-art unsupervised adaptation methods, effectively bridging the domain gap.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Pre-trained image restoration models often fail on real-world, out-of-distribution degradations due to significant domain gaps. Adapting to these unseen domains is challenging, as out-of-distribution data lacks ground truth, and traditional adaptation methods often require complex architectural changes. We propose LEGO (Learning from a Generative Oracle), a practical three-stage framework for post-training domain adaptation without paired data. LEGO converts this unsupervised challenge into a tractable pseudo-supervised one. First, we obtain initial restorations from the pre-trained model. Second, we leverage a frozen, large-scale generative oracle to refine these estimates into high-quality pseudo-ground-truths. Third, we fine-tune the original model using a mixed-supervision strategy combining in-distribution data with these new pseudo-pairs. This approach adapts the model to the new distribution without sacrificing its original robustness or requiring architectural modifications. Experiments demonstrate that LEGO effectively bridges the domain gap, significantly improving performance on diverse real-world benchmarks.

Problem

• Adapts pre-trained restoration models to unseen real-world degradations
• Converts unsupervised domain adaptation into pseudo-supervised learning
• Fine-tunes models using generative oracle without architectural changes

Innovation

• Three-stage framework for unsupervised domain adaptation
• Generative oracle refines estimates into pseudo-ground-truth
• Mixed-supervision fine-tuning without architectural changes