PromptGate Client Adaptive Vision Language Gating for Open Set Federated Active Learning

2026-03-07

AI Summary

This work addresses the challenge of out-of-distribution (OOD) noise in open-set scenarios within resource-constrained medical federated learning, where conventional active learning methods often misselect samples for annotation, wasting limited labeling resources. To mitigate this, the authors propose PromptGate, a framework that employs a client-adaptive vision-language gating mechanism with learnable class-specific prompt vectors to dynamically adapt the frozen BiomedCLIP model to local clinical domains. Integrated with FedAvg, PromptGate enables data-private OOD filtering and sample purification without sharing raw data. The approach supports policy-agnostic dynamic OOD detection and achieves over 95% in-distribution sample purity and more than 98% OOD recall on dermatological and breast imaging datasets, significantly outperforming static prompting baselines and enhancing both annotation efficiency and model robustness.

Technology Category

• Machine Learning: Active Learning
• Computer Vision: Diffusion Models for Vision
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Deploying medical AI across resource-constrained institutions demands data-efficient learning pipelines that respect patient privacy. Federated Learning (FL) enables collaborative medical AI without centralising data, yet real-world clinical pools are inherently open-set, containing out-of-distribution (OOD) noise such as imaging artifacts and wrong modalities. Standard Active Learning (AL) query strategies mistake this noise for informative samples, wasting scarce annotation budgets. We propose PromptGate, a dynamic VLM-gated framework for Open-Set Federated AL that purifies unlabeled pools before querying. PromptGate introduces a federated Class-Specific Context Optimization: lightweight, learnable prompt vectors that adapt a frozen BiomedCLIP backbone to local clinical domains and aggregate globally via FedAvg -- without sharing patient data. As new annotations arrive, prompts progressively sharpen the ID/OOD boundary, turning the VLM into a dynamic gatekeeper that is strategy-agnostic: a plug-and-play pre-selection module enhancing any downstream AL strategy. Experiments on distributed dermatology and breast imaging benchmarks show that while static VLM prompting degrades to 50% ID purity, PromptGate maintains $>$95% purity with 98% OOD recall.

Problem

• Open-Set Federated Learning
• Out-of-Distribution Noise
• Active Learning
• Medical AI
• Data Efficiency

Innovation

• PromptGate
• Federated Active Learning
• Vision-Language Model
• Open-Set Recognition
• Class-Specific Prompting