Privacy-Preserving Federated Vision Transformer Learning Leveraging Lightweight Homomorphic Encryption in Medical AI
To address the dual challenges of privacy leakage (e.g., gradient inversion attacks) and high communication overhead in medical federated learning, this paper proposes a privacy-enhancing framework integrating Vision Transformers (ViT) with CKKS homomorphic encryption. The core method replaces conventional encrypted gradient aggregation with homomorphically encrypted CLS tokens from ViT—enabling secure, feature-level aggregation and direct inference directly in the ciphertext domain. This design eliminates gradient reconstruction vulnerabilities while drastically reducing communication costs: per-round overhead drops by 30× to merely 326 KB. Evaluated on a lung cancer histopathological classification task, the framework achieves a global accuracy of 96.12%, with ciphertext-domain inference maintaining 90.02% accuracy. Thus, it simultaneously delivers strong privacy guarantees, substantial communication efficiency gains, and competitive model performance.