Text-Driven Fusion for Infrared and Visible Images: Achieving Image Scene Adaptation on Hyperbolic Space

2026-06-13

AI Summary

Existing methods operating in Euclidean space for infrared and visible image fusion often suffer from multimodal interaction distortion and disruption of semantic hierarchies due to rigid distance metrics. This work addresses these limitations by introducing hyperbolic manifolds into the fusion task for the first time, proposing a text-driven framework that leverages BLIP-derived textual prompts as topological anchors in hyperbolic space to guide cross-modal visual alignment. Notably, the model achieves adaptive fusion during inference without requiring textual input. By employing Poincaré ball-based hyperbolic embeddings, the approach effectively exploits negative curvature to model semantic hierarchies, thereby avoiding metric saturation and preserving fine texture details. Extensive experiments demonstrate that the proposed method significantly outperforms state-of-the-art techniques across multiple benchmarks, simultaneously enhancing both semantic consistency and visual quality of the fused images.

Abstract

Infrared and visible image fusion aims to integrate complementary modalities, while existing Euclidean methods impose rigid distance metrics that distort multi-modal interactions and parent-to-child semantic hierarchies. To overcome these limitations, we introduce a text-driven fusion framework empowered by hyperbolic manifold learning. During training, BLIP-extracted text prompts serve as topological anchors within the hyperbolic space, guiding vision-attribute alignment through hyperbolic embeddings that naturally accommodate varying semantic granularities. By exploiting the exponential volume growth dictated by the Poincaré ball's negative curvature, this approach seamlessly embeds hierarchical trees to encode coarse-to-fine semantics without metric saturation, while the vast peripheral space prevents texture distortion during cross-modal fusion. At inference, the fusion process autonomously adapts to input content using the learned text-attribute priors, completely eliminating the need for textual input. Experimental results show our method outperforms state-of-the-art approaches on benchmark datasets, with code available at https://github.com/Shaoyun2023/TEDFusion.

Problem

• infrared and visible image fusion
• Euclidean methods
• semantic hierarchies
• multi-modal interactions
• distance metrics

Innovation

• hyperbolic manifold learning
• text-driven fusion
• infrared-visible image fusion
• semantic hierarchy
• Poincaré ball