Enhancing Breast Cancer Detection with Vision Transformers and Graph Neural Networks

2025-07-11

AI Summary

To address the limited accuracy of early breast cancer detection—thereby constraining patient survival—this paper proposes a novel multimodal medical image analysis framework integrating Vision Transformers (ViT) and Graph Neural Networks (GNN). The method jointly models global semantic features and topological relationships among lesion regions on the CBIS-DDSM dataset, while incorporating an interpretable attention mechanism to generate clinically actionable heatmaps for radiologist decision support. Experimental results demonstrate an accuracy of 84.2%, significantly outperforming conventional CNNs and unimodal baselines. The core contribution lies in the first deep integration of ViT and GNN for relational reasoning over mammographic lesions, achieving both high discriminative performance and model transparency. This design bridges deep learning efficacy with clinical interpretability, offering clear translational potential for routine breast cancer screening.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Breast cancer is a leading cause of death among women globally, and early detection is critical for improving survival rates. This paper introduces an innovative framework that integrates Vision Transformers (ViT) and Graph Neural Networks (GNN) to enhance breast cancer detection using the CBIS-DDSM dataset. Our framework leverages ViT's ability to capture global image features and GNN's strength in modeling structural relationships, achieving an accuracy of 84.2%, outperforming traditional methods. Additionally, interpretable attention heatmaps provide insights into the model's decision-making process, aiding radiologists in clinical settings.

Problem

• Enhancing breast cancer detection accuracy using advanced deep learning
• Integrating Vision Transformers and Graph Neural Networks for medical imaging
• Improving interpretability of AI decisions for clinical diagnostics

Innovation

• Integrates Vision Transformers for global features
• Uses Graph Neural Networks for structural relationships
• Achieves 84.2% accuracy on CBIS-DDSM dataset