CheXLearner: Text-Guided Fine-Grained Representation Learning for Progression Detection

2025-05-11

AI Summary

To address semantic misalignment caused by coarse-grained image-text alignment and the lack of medical semantic guidance in purely vision-based approaches for longitudinal chest X-ray analysis, this paper proposes the first end-to-end multimodal framework integrating anatomical region localization, hyperbolic geometric modeling, and Riemannian manifold alignment. Its core innovation is the Med-Manifold Alignment Module (Med-MAM), which enables fine-grained, pathology-interpretable alignment of anatomical structures across time series—marking the first such effort at the clinical interpretability level. We further introduce multimodal fine-grained supervision and dynamic low-level feature optimization. On anatomical region progression detection, our method achieves 81.12% accuracy (+17.2%) and 80.32% F1-score (+11.05%). For downstream disease classification, it attains a mean AUC of 91.52%, significantly outperforming state-of-the-art methods.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Learning with Manifolds
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Temporal medical image analysis is essential for clinical decision-making, yet existing methods either align images and text at a coarse level - causing potential semantic mismatches - or depend solely on visual information, lacking medical semantic integration. We present CheXLearner, the first end-to-end framework that unifies anatomical region detection, Riemannian manifold-based structure alignment, and fine-grained regional semantic guidance. Our proposed Med-Manifold Alignment Module (Med-MAM) leverages hyperbolic geometry to robustly align anatomical structures and capture pathologically meaningful discrepancies across temporal chest X-rays. By introducing regional progression descriptions as supervision, CheXLearner achieves enhanced cross-modal representation learning and supports dynamic low-level feature optimization. Experiments show that CheXLearner achieves 81.12% (+17.2%) average accuracy and 80.32% (+11.05%) F1-score on anatomical region progression detection - substantially outperforming state-of-the-art baselines, especially in structurally complex regions. Additionally, our model attains a 91.52% average AUC score in downstream disease classification, validating its superior feature representation.

Problem

• Detects anatomical region progression in temporal chest X-rays
• Aligns images and text at fine-grained semantic level
• Integrates medical semantics with visual information

Innovation

• End-to-end framework for anatomical region detection
• Hyperbolic geometry for robust structure alignment
• Regional progression descriptions for cross-modal learning