Zero-Training Task-Specific Model Synthesis for Few-Shot Medical Image Classification

2025-11-17

AI Summary

Medical image classification faces severe data scarcity in scenarios such as rare diseases, where labeled samples are extremely limited. This paper introduces the first zero-training, task-specific model synthesis framework: given only a single medical image and its corresponding clinical text description, it end-to-end synthesizes the complete weight parameters of a dedicated classifier (e.g., EfficientNet-V2), requiring no fine-tuning or additional training. The core innovation lies in a semantics-guided generative engine that directly maps multimodal task information—image and clinical text—into model parameters, bypassing conventional paradigms reliant on large-scale pretraining or few-shot adaptation. Evaluated on ISIC 2018 and rare-disease benchmarks, our method achieves state-of-the-art 1-shot and 5-shot classification accuracy, outperforming existing zero-shot and few-shot approaches. This work establishes a novel paradigm for deploying resource-efficient, low-data medical AI systems.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Deep learning models have achieved remarkable success in medical image analysis but are fundamentally constrained by the requirement for large-scale, meticulously annotated datasets. This dependency on "big data" is a critical bottleneck in the medical domain, where patient data is inherently difficult to acquire and expert annotation is expensive, particularly for rare diseases where samples are scarce by definition. To overcome this fundamental challenge, we propose a novel paradigm: Zero-Training Task-Specific Model Synthesis (ZS-TMS). Instead of adapting a pre-existing model or training a new one, our approach leverages a large-scale, pre-trained generative engine to directly synthesize the entire set of parameters for a task-specific classifier. Our framework, the Semantic-Guided Parameter Synthesizer (SGPS), takes as input minimal, multi-modal task information as little as a single example image (1-shot) and a corresponding clinical text description to directly synthesize the entire set of parameters for a task-specific classifier. The generative engine interprets these inputs to generate the weights for a lightweight, efficient classifier (e.g., an EfficientNet-V2), which can be deployed for inference immediately without any task-specific training or fine-tuning. We conduct extensive evaluations on challenging few-shot classification benchmarks derived from the ISIC 2018 skin lesion dataset and a custom rare disease dataset. Our results demonstrate that SGPS establishes a new state-of-the-art, significantly outperforming advanced few-shot and zero-shot learning methods, especially in the ultra-low data regimes of 1-shot and 5-shot classification. This work paves the way for the rapid development and deployment of AI-powered diagnostic tools, particularly for the long tail of rare diseases where data is critically limited.

Problem

• Overcoming data scarcity in medical image classification
• Synthesizing task-specific models without training data
• Enabling AI diagnosis for rare diseases with minimal samples

Innovation

• Directly synthesizes classifier parameters using generative engine
• Uses single image and text description for model generation
• Deploys synthesized models without task-specific training