Abn-BLIP: Abnormality-aligned Bootstrapping Language-Image Pre-training for Pulmonary Embolism Diagnosis and Report Generation from CTPA

2025-03-03

AI Summary

To address incomplete abnormality detection, misalignment between visual and textual semantics, and insufficient report structuring in pulmonary embolism (PE) diagnosis and reporting from CT pulmonary angiography (CTPA) images, this paper proposes an abnormality-aligned bootstrapped vision-language pretraining framework. Methodologically, it integrates a 3D vision encoder with a learnable abnormality query module to establish a dual-path cross-modal attention mechanism, enabling fine-grained alignment between lesion regions and corresponding textual descriptions; a structured decoder further generates standardized radiology reports. This work introduces the first bootstrapped, abnormality-aware vision-language co-pretraining paradigm specifically designed for medical imaging. Evaluated on a multi-center CTPA dataset, the framework reduces PE missed-diagnosis rate by 21.4% and improves clinical relevance score by 18.7%, significantly outperforming existing state-of-the-art models.

Technology Category

• Computer Vision: Multi-modal Vision
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Medical imaging plays a pivotal role in modern healthcare, with computed tomography pulmonary angiography (CTPA) being a critical tool for diagnosing pulmonary embolism and other thoracic conditions. However, the complexity of interpreting CTPA scans and generating accurate radiology reports remains a significant challenge. This paper introduces Abn-BLIP (Abnormality-aligned Bootstrapping Language-Image Pretraining), an advanced diagnosis model designed to align abnormal findings to generate the accuracy and comprehensiveness of radiology reports. By leveraging learnable queries and cross-modal attention mechanisms, our model demonstrates superior performance in detecting abnormalities, reducing missed findings, and generating structured reports compared to existing methods. Our experiments show that Abn-BLIP outperforms state-of-the-art medical vision-language models and 3D report generation methods in both accuracy and clinical relevance. These results highlight the potential of integrating multimodal learning strategies for improving radiology reporting. The source code is available at https://github.com/zzs95/abn-blip.

Problem

• Improves CTPA scan interpretation for pulmonary embolism diagnosis.
• Enhances accuracy and comprehensiveness of radiology report generation.
• Reduces missed findings in thoracic condition detection.

Innovation

• Abnormality-aligned Bootstrapping Language-Image Pretraining
• Learnable queries and cross-modal attention mechanisms
• Superior performance in detecting abnormalities and report generation