Faster and Better: Reinforced Collaborative Distillation and Self-Learning for Infrared-Visible Image Fusion

2025-09-02

AI Summary

To address the performance limitations of lightweight models in visible-infrared image fusion, this paper proposes a reinforcement learning (RL)-guided collaborative distillation framework. The method jointly optimizes knowledge transfer and model self-improvement by dynamically modulating teacher guidance strength, generating hard-negative samples, and integrating a self-learning mechanism. Innovatively, RL, knowledge distillation, and self-learning are embedded into a lightweight network architecture without increasing parameter count, thereby significantly enhancing multimodal complementary information utilization. Experimental results on mainstream benchmarks demonstrate that the proposed approach achieves average improvements of over 3.2% in PSNR and SSIM, while attaining an inference speed of 42 FPS—substantially outperforming existing lightweight fusion methods. The framework thus effectively balances high-fidelity fusion quality with real-time processing requirements.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Infrared and visible image fusion plays a critical role in enhancing scene perception by combining complementary information from different modalities. Despite recent advances, achieving high-quality image fusion with lightweight models remains a significant challenge. To bridge this gap, we propose a novel collaborative distillation and self-learning framework for image fusion driven by reinforcement learning. Unlike conventional distillation, this approach not only enables the student model to absorb image fusion knowledge from the teacher model, but more importantly, allows the student to perform self-learning on more challenging samples to enhance its capabilities. Particularly, in our framework, a reinforcement learning agent explores and identifies a more suitable training strategy for the student.The agent takes both the student's performance and the teacher-student gap as inputs, which leads to the generation of challenging samples to facilitate the student's self-learning. Simultaneously, it dynamically adjusts the teacher's guidance strength based on the student's state to optimize the knowledge transfer. Experimental results demonstrate that our method can significantly improve student performance and achieve better fusion results compared to existing techniques.

Problem

• Achieving high-quality infrared-visible image fusion with lightweight models
• Enabling student models to absorb knowledge and self-learn on challenging samples
• Dynamically optimizing knowledge transfer through reinforcement learning guidance

Innovation

• Reinforcement learning-driven collaborative distillation framework
• Self-learning on challenging samples for student enhancement
• Dynamic adjustment of teacher guidance strength