CWCD: Category-Wise Contrastive Decoding for Structured Medical Report Generation

2026-04-11

AI Summary

This work addresses the tendency of existing medical image report generation methods to over-rely on linguistic priors, often producing hallucinated pathological descriptions inconsistent with visual content. To mitigate this issue, the authors propose CWCD, a modular framework that introduces category-level parameterization and contrastive decoding for structured chest X-ray report generation. By leveraging category-specific visual prompts to contrast normal and masked images, CWCD enhances attention to critical pathological features and alleviates visual attention decay during single-pass generation. Built upon a multimodal large language model, the approach integrates structured report generation, visual prompt guidance, and contrastive learning. Experimental results demonstrate significant improvements over current baselines across multiple clinical validity and text generation metrics, with ablation studies confirming the contribution of each component.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Interpreting chest X-rays is inherently challenging due to the overlap between anatomical structures and the subtle presentation of many clinically significant pathologies, making accurate diagnosis time-consuming even for experienced radiologists. Recent radiology-focused foundation models, such as LLaVA-Rad and Maira-2, have positioned multi-modal large language models (MLLMs) at the forefront of automated radiology report generation (RRG). However, despite these advances, current foundation models generate reports in a single forward pass. This decoding strategy diminishes attention to visual tokens and increases reliance on language priors as generation proceeds, which in turn introduces spurious pathology co-occurrences in the generated reports. To mitigate these limitations, we propose Category-Wise Contrastive Decoding (CWCD), a novel and modular framework designed to enhance structured radiology report generation (SRRG). Our approach introduces category-specific parameterization and generates category-wise reports by contrasting normal X-rays with masked X-rays using category-specific visual prompts. Experimental results demonstrate that CWCD consistently outperforms baseline methods across both clinical efficacy and natural language generation metrics. An ablation study further elucidates the contribution of each architectural component to overall performance.

Problem

• radiology report generation
• spurious pathology co-occurrences
• visual-language alignment
• structured medical reporting
• multi-modal large language models

Innovation

• Category-Wise Contrastive Decoding
• Structured Radiology Report Generation
• Multi-modal Large Language Models
• Visual Prompting
• Pathology Co-occurrence Mitigation