AEGIS: A Multi-Task Joint-Embedding Predictive Architecture for Mammography

2026-06-30

AI Summary

This work addresses two key clinical tasks in mammography—breast cancer detection and breast density assessment—through a unified modeling framework. It introduces, for the first time, a self-supervised Joint Embedding Predictive Architecture (JEPA) for multi-task learning on mammographic images, leveraging Vision Transformer backbones (Small/Base/Large) pretrained without manual annotations. The approach incorporates progressive high-resolution fine-tuning and model ensembling, achieving strong cross-task generalization and zero-shot transferability across diverse populations. On an internal test set, the model attains an AUC of 0.949 for cancer triage, 0.953 for binary density classification, and 62.6% accuracy for four-class BI-RADS categorization. Under a zero-shot setting on the VinDr-Mammo dataset, it achieves an AUC of 0.871, demonstrating robust out-of-distribution performance.

Abstract

We present Aegis, a joint-embedding predictive architecture for breast cancer detection and density assessment in mammography. We train three Vision Transformer variants (Small/Base/Large) using self-supervised joint-embedding predictive architecture (JEPA) pre-training on 71,103 studies from 14 clinical sites, followed by supervised fine-tuning with progressive resolution scaling up to 2048x1536. On a curated 785-study test set, our largest model achieves area under the receiver operating characteristic curve (AUC) 0.949 for breast cancer triage with 93% sensitivity and 75% specificity at the optimal operating point. An ensemble combining our model with a U.S. Food and Drug Administration-cleared baseline further improves discrimination to 0.952 AUC. For breast density classification, the model achieves 0.953 AUC for binary (dense vs. non-dense) classification and 62.6% exact accuracy across four Breast Imaging Reporting and Data System (BI-RADS) categories, with 98.8% adjacent accuracy comparable to reported human inter-reader agreement. External validation on the public VinDr-Mammo dataset provides evidence of cross-population transfer under a different reference standard, with the largest model achieving 0.871 AUC for triage in a zero-shot setting.

Problem

• breast cancer detection
• breast density assessment
• mammography
• multi-task learning

Innovation

• joint-embedding predictive architecture
• Vision Transformer
• self-supervised pre-training
• progressive resolution scaling
• multi-task mammography analysis