CCRC: A Change-Aware Captioning and Reasoning Chain for Image Change Captioning and Segmentation

2026-06-27

AI Summary

This work addresses the limited spatial localization capability of existing image change captioning methods, which struggle to accurately describe fine-grained differences between image pairs. To overcome this, the paper introduces a novel task—Image Change Captioning and Segmentation (ICCS)—and proposes a dual-chain decoupled framework, CCRC, which jointly generates structured semantic descriptions and pixel-level change segmentation for the first time. The framework incorporates multi-head change-aware attention, a visual fusion mechanism, a change-aware token refiner, and a multimodal large language model to dynamically identify and process segmentable change regions. Experimental results demonstrate that the proposed method achieves state-of-the-art performance under pixel-level supervision on both synthetic and real-world change detection benchmarks.

Abstract

Understanding and localizing subtle changes between paired images is critical for tasks such as surveillance and image editing. However, traditional Image Change Captioning (ICC) methods lack spatial grounding, limiting their precision. We introduce Image Change Captioning and Segmentation (ICCS), a new multimodal task that jointly requires structured change description and pixel-level localization. To address ICCS, we propose the Change-aware Captioning and Reasoning Chain (CCRC), a dual-chain framework that decouples semantic reasoning from spatial segmentation. The first chain, Chain-of-Change-Captioning (CCC), enhances fine-grained change perception via a visual fusion module based on Multi-Head Change-aware Attention inserted between the visual and language components of a Multimodal Large Language Model (MLLM). CCC also determines whether a change is segmentable. If not, it alone generates the caption. Otherwise, the second chain, Chain-of-Change-Segmenting (CCS), is activated, leveraging spatial priors from CCC and refining masks with a Change-aware Token Refiner for accurate boundary localization. We evaluate CCRC on both synthetic and real-world change detection benchmarks with pixel-level supervision. Experiments show CCRC achieves state-of-the-art performance.

Problem

• Image Change Captioning
• Change Segmentation
• Spatial Grounding
• Multimodal Task
• Pixel-level Localization

Innovation

• Change-aware Attention
• Multimodal Large Language Model
• Image Change Captioning and Segmentation
• Chain-of-Reasoning
• Token Refiner