Brain-Adapter: A Dual-Stream Vision-Language MIL Framework for Comprehensive 3D CT Diagnosis of Acute Intracranial Pathologies

2026-06-22

AI Summary

This study addresses the limitations of dense manual annotations and insufficient semantic understanding in existing approaches for automated 3D brain CT diagnosis. The authors propose a dual-stream multiple instance learning framework that, for the first time, enables end-to-end alignment of 3D pathological semantics guided directly by raw radiology reports. The method integrates a pretrained 2D biomedical vision–language model with textual information, incorporating a text-conditioned attention mechanism and an uncertainty-aware refinement module, while enforcing representation consistency to coordinate predictions across both streams. Additionally, it leverages a large language model to structurally extract diagnostic labels from reports. Evaluated on multi-label classification of acute intracranial abnormalities, the approach significantly outperforms current 3D models and standard MIL baselines, achieving high diagnostic accuracy and strong clinical scalability.

Abstract

Automated diagnosis of 3D brain CT scans is essential for critical care, yet it remains challenging due to the heavy reliance on manual annotations and the limited semantic understanding of conventional models. While 2D foundation vision-language models (VLMs) have shown remarkable generalization, effectively transferring their representational power to 3D volumes remains an open problem. In this paper, we propose Brain-Adapter, a novel dual-stream multiple instance learning (MIL) framework that leverages pre-trained 2D biomedical VLMs and raw diagnostic reports for robust scan-level multi-label classification. Specifically, we introduce a Text-Conditioned Attention (TCA) mechanism, utilizing raw diagnostic sentences as semantic queries to dynamically align visual cues with specific disease concepts. Concurrently, a parallel visual MIL stream captures global scan characteristics, supervised by structured labels extracted via a Large Language Model (LLM). To ensure representation coherence, a consistency constraint enforces synergy between the two streams. During inference, an Uncertainty-Aware Refinement (UAR) module dynamically calibrates and fuses these dual-stream predictions to resolve ambiguous cases. Extensive experiments demonstrate that our method significantly outperforms state-of-the-art 3D models and standard MIL approaches. By eliminating the reliance on dense annotations, Brain-Adapter provides a highly scalable and clinically viable solution for 3D acute intracranial pathology analysis.

Problem

• 3D CT diagnosis
• acute intracranial pathologies
• vision-language models
• multiple instance learning
• automated medical diagnosis

Innovation

• Vision-Language Model
• Multiple Instance Learning
• 3D Medical Imaging
• Text-Conditioned Attention
• Uncertainty-Aware Refinement