Trust but Verify: Adaptive Conditioning for Reference-Based Diffusion Super-Resolution via Implicit Reference Correlation Modeling

2026-02-02

AI Summary

This work addresses the challenge of unreliable correspondences between low-quality images and reference images under real-world degradation conditions, which often leads to misuse or underutilization of reference information. To this end, the authors propose Ada-RefSR, a single-step diffusion framework that adaptively modulates reference guidance based on the principle of “trust but verify”—effectively leveraging the reference when it is reliable and suppressing it otherwise. The core innovation lies in the Adaptive Implicit Correlation Gating (AICG) mechanism, which implicitly models inter-image correlations through learnable summary tokens, enabling lightweight yet robust reference fusion. Extensive experiments demonstrate that the proposed method consistently achieves a favorable balance among fidelity, naturalness, and efficiency across multiple datasets, while maintaining robust performance under varying reference alignment conditions.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Adversarial Learning & Robustness
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Recent works have explored reference-based super-resolution (RefSR) to mitigate hallucinations in diffusion-based image restoration. A key challenge is that real-world degradations make correspondences between low-quality (LQ) inputs and reference (Ref) images unreliable, requiring adaptive control of reference usage. Existing methods either ignore LQ-Ref correlations or rely on brittle explicit matching, leading to over-reliance on misleading references or under-utilization of valuable cues. To address this, we propose Ada-RefSR, a single-step diffusion framework guided by a"Trust but Verify"principle: reference information is leveraged when reliable and suppressed otherwise. Its core component, Adaptive Implicit Correlation Gating (AICG), employs learnable summary tokens to distill dominant reference patterns and capture implicit correlations with LQ features. Integrated into the attention backbone, AICG provides lightweight, adaptive regulation of reference guidance, serving as a built-in safeguard against erroneous fusion. Experiments on multiple datasets demonstrate that Ada-RefSR achieves a strong balance of fidelity, naturalness, and efficiency, while remaining robust under varying reference alignment.

Problem

• reference-based super-resolution
• diffusion models
• image restoration
• implicit correlation
• reference reliability

Innovation

• Adaptive Implicit Correlation Gating
• Reference-Based Super-Resolution
• Diffusion Models
• Implicit Correlation Modeling
• Trust but Verify