DFM: Difference Feature Modeling with Text-Guided Gated Contrastive Loss for Remote Sensing Image Change Captioning

2026-06-25

AI Summary

This work addresses the limitations of existing remote sensing image change captioning methods, which rely on a single autoregressive generation paradigm and tend to favor frequent vocabulary while overlooking discriminative differences. To overcome this, the authors propose a difference-aware feature modeling framework that leverages a text-guided gated contrastive loss to steer the visual encoder—via linguistic cues—toward salient change regions. The approach integrates a pretrained change detection model with a multi-scale joint feature modeling module to comprehensively capture spatiotemporal discrepancies between multi-temporal images. By transcending the constraints of conventional generative paradigms, the method achieves significant improvements in both caption accuracy and discriminative expression across multiple remote sensing change captioning benchmarks, demonstrating its effectiveness and novelty.

Abstract

The primary goal of Remote Sensing Image Change Captioning (RSICC) is to automatically generate descriptions of changes between remote sensing images captured at different time points. Existing models still rely on a single autoregressive generation paradigm, which tends to prioritize learning easily generated vocabulary over capturing discriminative differences between images. To address this, we reframe the training paradigm and propose a novel Difference Feature Modeling (DFM) framework. Specifically, we introduce a Text-guided Gated Contrastive Loss (TGCL) to guide the vision encoder to extract critical features from a text-modal perspective. Additionally, we incorporate a pre-trained Change Detection model to transfer stable change detection knowledge. In order to further enhance the representation, we design a Joint Feature Modeling (JFM) module to achieve the fusion of multi-scale difference representations, thereby capturing comprehensive spatiotemporal variations between multi-temporal images. Extensive experiments on multiple datasets demonstrate the effectiveness of our approach.

Problem

• Remote Sensing Image Change Captioning
• Change Description Generation
• Discriminative Difference Modeling
• Multi-temporal Image Analysis

Innovation

• Difference Feature Modeling
• Text-Guided Gated Contrastive Loss
• Remote Sensing Image Change Captioning
• Joint Feature Modeling
• Change Detection