MAVFusion: Efficient Infrared and Visible Video Fusion via Motion-Aware Sparse Interaction

2026-04-02

AI Summary

Existing methods for infrared and visible-light video fusion struggle to simultaneously achieve temporal consistency and computational efficiency. To address this challenge, this work proposes MAVFusion, an end-to-end video fusion framework that introduces a novel optical flow-guided dynamic region detection mechanism to decouple processing of dynamic and static regions. Specifically, dense cross-modal attention is applied in dynamic regions to preserve fine details, while a lightweight weak-interaction module is employed in static regions to enhance efficiency. By adaptively sparsifying cross-modal interactions, the proposed method achieves state-of-the-art performance across multiple benchmarks, attaining a real-time inference speed of 14.16 FPS at 640×480 resolution—significantly outperforming existing approaches.

Abstract

Infrared and visible video fusion combines the object saliency from infrared images with the texture details from visible images to produce semantically rich fusion results. However, most existing methods are designed for static image fusion and cannot effectively handle frame-to-frame motion in videos. Current video fusion methods improve temporal consistency by introducing interactions across frames, but they often require high computational cost. To mitigate these challenges, we propose MAVFusion, an end-to-end video fusion framework featuring a motion-aware sparse interaction mechanism that enhances efficiency while maintaining superior fusion quality. Specifically, we leverage optical flow to identify dynamic regions in multi-modal sequences, adaptively allocating computationally intensive cross-modal attention to these sparse areas to capture salient transitions and facilitate inter-modal information exchange. For static background regions, a lightweight weak interaction module is employed to maintain structural and appearance integrity. By decoupling the processing of dynamic and static regions, MAVFusion simultaneously preserves temporal consistency and fine-grained details while significantly accelerating inference. Extensive experiments demonstrate that MAVFusion achieves state-of-the-art performance on multiple infrared and visible video benchmarks, achieving a speed of 14.16\,FPS at $640 \times 480$ resolution. The source code will be available at https://github.com/ixilai/MAVFusion.

Problem

• infrared and visible video fusion
• temporal consistency
• motion handling
• computational efficiency

Innovation

• motion-aware sparse interaction
• infrared-visible video fusion
• optical flow
• cross-modal attention
• temporal consistency