GTA-Net: Cooperative Game Theory for Vision-Language Alignment in Chest X-Ray Report Generation

2026-06-20

AI Summary

Current vision-language models for chest X-ray report generation lack explicit region-word correspondence and disease-level consistency, leading to clinically unreliable descriptions. This work introduces cooperative game theory into medical vision-language generation for the first time, proposing BinaryGameAligner—a framework that leverages Shapley values to weight fine-grained interactions between image regions and textual tokens. It further incorporates a disease-aware triplet aligner to integrate structured disease concepts, enabling cross-modal semantic alignment. Combining a Swin visual encoder, a large language model fine-tuned with LoRA, and a joint optimization objective, the method achieves significant improvements in both generation quality and clinical consistency on CheXpertPlus and IU-XRay, outperforming existing approaches.

Abstract

Automated chest X-ray report generation requires precise cross-modal grounding to ensure clinically reliable descriptions. However, existing vision-language models rely on implicit attention mechanisms that fail to enforce explicit region-word correspondence and disease-level consistency. We propose Game-Theoretic Alignment Network (GTA-Net), a vision-language framework that formulates report generation as a cooperative game-theoretic alignment problem. The model introduces a BinaryGameAligner that models interactions between image regions and text tokens using similarity-based payoff matrices with Shapley-inspired importance weighting. To enforce clinical semantics, we further develop a Disease-Aware Ternary Aligner, which captures joint interactions among images, reports, and structured disease concepts. GTA-Net combines a Swin-based visual encoder with a LoRA-adapted large language model and is trained with a unified objective for generation and alignment. Experiments on CheXpertPlus and IU-XRay demonstrate state-of-the-art performance across standard generation metrics and improved clinical consistency, highlighting the effectiveness of explicit game-theoretic alignment for medical vision-language generation.

Problem

• vision-language alignment
• chest X-ray report generation
• cross-modal grounding
• clinical consistency
• region-word correspondence

Innovation

• Game-Theoretic Alignment
• Vision-Language Alignment
• Chest X-Ray Report Generation
• Shapley Value
• Disease-Aware Modeling