Welcome New Doctor: Continual Learning with Expert Consultation and Autoregressive Inference for Whole Slide Image Analysis

2025-08-04

AI Summary

In whole-slide image (WSI) analysis, continual learning faces critical challenges: excessive storage overhead, reliance on historical data replay, and severe inter-task interference. To address these, we propose COSFormer—a lightweight Transformer-based incremental learning framework. COSFormer introduces an expert consultation mechanism and autoregressive inference to enable cross-task knowledge transfer, while integrating parameter isolation with a sparse mixture-of-experts architecture to support both class-incremental and task-incremental learning without retraining previous task models. Evaluated on seven cross-organ WSI datasets, COSFormer consistently outperforms state-of-the-art continual learning methods, achieving superior accuracy while significantly reducing computational and memory costs. Its design ensures strong generalization across diverse tissue types and practical feasibility for clinical deployment.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Learning & Optimization for CV
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Whole Slide Image (WSI) analysis, with its ability to reveal detailed tissue structures in magnified views, plays a crucial role in cancer diagnosis and prognosis. Due to their giga-sized nature, WSIs require substantial storage and computational resources for processing and training predictive models. With the rapid increase in WSIs used in clinics and hospitals, there is a growing need for a continual learning system that can efficiently process and adapt existing models to new tasks without retraining or fine-tuning on previous tasks. Such a system must balance resource efficiency with high performance. In this study, we introduce COSFormer, a Transformer-based continual learning framework tailored for multi-task WSI analysis. COSFormer is designed to learn sequentially from new tasks wile avoiding the need to revisit full historical datasets. We evaluate COSFormer on a sequence of seven WSI datasets covering seven organs and six WSI-related tasks under both class-incremental and task-incremental settings. The results demonstrate COSFormer's superior generalizability and effectiveness compared to existing continual learning frameworks, establishing it as a robust solution for continual WSI analysis in clinical applications.

Problem

• Efficient continual learning for large Whole Slide Images
• Adapting models to new tasks without retraining
• Balancing resource efficiency with high performance

Innovation

• Transformer-based continual learning framework
• Autoregressive inference for multi-task analysis
• Efficient learning without revisiting historical data