Dream-IF: Dynamic Relative EnhAnceMent for Image Fusion

2025-03-13

AI Summary

Image fusion quality often degrades due to inconsistent sensor-specific degradations across modalities, and conventional methods decouple enhancement from fusion, neglecting intrinsic inter-modal correlations. To address this, we propose a novel paradigm—“dominant-region-driven dynamic relative enhancement”—that jointly optimizes fusion and enhancement for the first time. Our approach introduces a fusion-guided relative dominance metric, establishes a cross-modal bidirectional enhancement framework, and incorporates degradation-aware prompt encoding with recursive cross-modal attention to enable dynamic, adaptive, and mutually reinforcing restoration. Extensive experiments on infrared–visible light and medical multimodal fusion tasks demonstrate state-of-the-art performance: average improvements of 1.8 dB in PSNR and 0.023 in SSIM over prior methods, with significant gains in detail preservation and structural consistency.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Image fusion aims to integrate comprehensive information from images acquired through multiple sources. However, images captured by diverse sensors often encounter various degradations that can negatively affect fusion quality. Traditional fusion methods generally treat image enhancement and fusion as separate processes, overlooking the inherent correlation between them; notably, the dominant regions in one modality of a fused image often indicate areas where the other modality might benefit from enhancement. Inspired by this observation, we introduce the concept of dominant regions for image enhancement and present a Dynamic Relative EnhAnceMent framework for Image Fusion (Dream-IF). This framework quantifies the relative dominance of each modality across different layers and leverages this information to facilitate reciprocal cross-modal enhancement. By integrating the relative dominance derived from image fusion, our approach supports not only image restoration but also a broader range of image enhancement applications. Furthermore, we employ prompt-based encoding to capture degradation-specific details, which dynamically steer the restoration process and promote coordinated enhancement in both multi-modal image fusion and image enhancement scenarios. Extensive experimental results demonstrate that Dream-IF consistently outperforms its counterparts.

Problem

• Integrates comprehensive information from multi-source images
• Addresses degradations affecting fusion quality in diverse sensors
• Combines image enhancement and fusion for improved results

Innovation

• Dynamic Relative EnhAnceMent for Image Fusion
• Quantifies relative dominance across image layers
• Prompt-based encoding for degradation-specific restoration