FUMO: Prior-Modulated Diffusion for Single Image Reflection Removal

2026-03-19

AI Summary

This work addresses the challenge of reflection removal from a single image, where spatially varying reflection and transmission layers are highly coupled. To tackle this, we propose a prior-modulated diffusion framework that leverages dual priors—intensity and high-frequency—to explicitly guide the diffusion process. A coarse-to-fine two-stage training strategy is employed to inject conditional residuals and refine fine-scale details progressively. The framework integrates multi-scale residual aggregation, conditional residual gating, and a spatially adaptive refinement network. Extensive experiments demonstrate state-of-the-art quantitative performance and visually superior results on both standard benchmarks and complex real-world scenes, achieving an effective balance between structural fidelity and spatial controllability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Algorithmic accountability and transparency on the web
• Security and Privacy: Data transparency and provenance

Abstract

Single image reflection removal (SIRR) is challenging in real scenes, where reflection strength varies spatially and reflection patterns are tightly entangled with transmission structures. This paper presents a diffusion model with prior modulation framework (FUMO) that introduces explicit guidance signals to improve spatial controllability and structural faithfulness. Two priors are extracted directly from the mixed image, an intensity prior that estimates spatial reflection severity and a high-frequency prior that captures detail-sensitive responses via multi-scale residual aggregation. We propose a coarse-to-fine training paradigm. In the first stage, these cues are combined to gate the conditional residual injections, focusing the conditioning on regions that are both reflection-dominant and structure-sensitive. In the second stage, a fine-grained refinement network corrects local misalignment and sharpens fine details in the image space. Experiments conducted on both standard benchmarks and challenging images in the wild demonstrate competitive quantitative results and consistently improved perceptual quality. The code is released at https://github.com/Lucious-Desmon/FUMO.

Problem

• single image reflection removal
• spatially varying reflection
• reflection-transmission entanglement
• real-world reflection removal

Innovation

• prior-modulated diffusion
• single image reflection removal
• intensity prior
• high-frequency prior
• coarse-to-fine training