SDFed: Bridging Local Global Discrepancy via Subspace Refinement and Divergence Control in Federated Prompt Learning

2026-02-09

AI Summary

This work addresses the challenge of balancing client heterogeneity and privacy in federated prompt learning, where existing methods suffer from conflicts between local and global knowledge due to enforced uniformity in prompt structure and length. To overcome this limitation, we propose SDFed, a novel framework that maintains a fixed-length global prompt for efficient aggregation while allowing clients to learn variable-length local prompts tailored to their data distributions and resource constraints. SDFed integrates a subspace refinement mechanism and a divergence control strategy to harmonize local and global representations, thereby preserving informative content and enhancing knowledge transfer efficiency. As the first federated prompt learning architecture supporting heterogeneous prompt lengths, SDFed demonstrates significant improvements in performance, robustness, and communication efficiency across multiple benchmarks.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Prompt Engineering / Prompting
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing

Abstract

Vision-language pretrained models offer strong transferable representations, yet adapting them in privacy-sensitive multi-party settings is challenging due to the high communication cost of federated optimization and the limited local data on clients. Federated prompt learning mitigates this issue by keeping the VLPM backbone frozen and collaboratively training lightweight prompt parameters. However, existing approaches typically enforce a unified prompt structure and length across clients, which is inadequate under practical client heterogeneity in both data distributions and system resources, and may further introduce conflicts between globally shared and locally optimal knowledge. To address these challenges, we propose \textbf{SDFed}, a heterogeneous federated prompt learning framework that bridges Local-Global Discrepancy via Subspace Refinement and Divergence Control. SDFed maintains a fixed-length global prompt for efficient aggregation while allowing each client to learn a variable-length local prompt to better match its data characteristics and capacity. To mitigate local-global conflicts and facilitate effective knowledge transfer, SDFed introduces a subspace refinement method for local prompts and an information retention and divergence control strategy that preserves key local information while maintaining appropriate separability between global and local representations. Extensive experiments on several datasets demonstrate that SDFed consistently improves performance and robustness in heterogeneous federated settings.

Problem

• federated learning
• prompt learning
• client heterogeneity
• vision-language models
• local-global discrepancy

Innovation

• Federated Prompt Learning
• Heterogeneous Clients
• Subspace Refinement
• Divergence Control
• Vision-Language Models