Multi-View Synergistic Learning with Vision-Language Adaption for Low-Resource Biomedical Image Classification

2026-04-26

AI Summary

This work addresses the challenges of scarce annotations, subtle inter-class visual differences, and complex disease semantics in low-resource biomedical image classification by proposing a multi-view collaborative learning framework. The approach decouples the adaptation mechanisms of visual and textual encoders, incorporates multi-granularity contrastive learning, and integrates structured semantic supervision generated by large language models to achieve fine-grained and semantically consistent cross-modal representations. Evaluated across 11 public datasets spanning nine imaging modalities and ten anatomical regions, the method significantly outperforms state-of-the-art approaches under both few-shot and zero-shot settings, demonstrating its parameter efficiency and robust cross-modal alignment.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Accurate biomedical image classification under low-resource conditions remains challenging due to limited annotations, subtle inter-class visual differences, and complex disease semantics. While vision--language models offer a promising foundation for mitigating data scarcity, their effective adaptation in biomedical settings is constrained by the need for parameter-efficient tuning alongside fine-grained and semantically consistent representation learning. In this work, we propose Multi-View Synergistic Learning (MVSL), a unified framework that addresses these challenges by jointly considering adaptation paradigms, representation granularity, and disease semantic relationships. MVSL decouples the adaptation of visual and textual encoders to respect their distinct representational characteristics, enabling more stable and effective parameter-efficient fine-tuning. It further introduces multi-granularity contrastive learning to explicitly model both global image semantics and localized lesion-level evidence, improving fine-grained discrimination for visually similar disease categories. In addition, MVSL preserves disease-level semantic structure by incorporating structured supervision derived from large language models, which constrains textual representations at the class level and indirectly regularizes visual embeddings through cross-modal alignment. Together, these components enable more stable cross-modal alignment and improved discrimination under limited supervision. Extensive experiments on $11$ public biomedical datasets spanning $9$ imaging modalities and $10$ anatomical regions demonstrate that MVSL consistently outperforms state-of-the-art methods in few-shot and zero-shot classification settings.

Problem

• low-resource
• biomedical image classification
• vision-language models
• fine-grained discrimination
• semantic consistency

Innovation

• Multi-View Synergistic Learning
• Vision-Language Adaptation
• Parameter-Efficient Fine-Tuning
• Multi-Granularity Contrastive Learning
• Structured Semantic Supervision