SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models

2026-04-17

AI Summary

This work addresses the multifaceted security challenges confronting large language models in high-stakes scenarios—including privacy leakage, adversarial attacks, misinformation, and insufficient robustness—for which no unified solution currently exists. The authors propose SafeLM, a novel framework that, for the first time, cohesively integrates privacy preservation, attack defense, factual consistency, and robust aggregation within a federated learning paradigm. Through key technical innovations such as gradient-aware optimization, Paillier encryption, contrastive-guided calibrated decoding, and alignment-aware binary aggregation, SafeLM achieves state-of-the-art performance: it attains 98.0% accuracy in harmful content detection, reduces communication overhead by 96.9%, and suppresses gradient inversion attacks to a PSNR of 15.1 dB, thereby substantially enhancing model security, efficiency, and trustworthiness.

Technology Category

• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models

Application Category

• Security and Privacy: Security and privacy of machine learning and AI applications
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains lacking. We present SafeLM, a framework that jointly addresses four pillars of LLM safety: privacy, security, misinformation, and adversarial robustness. SafeLM combines federated training with gradient smartification and Paillier encryption for privacy, integrates defenses against training and inference-time attacks, employs contrastive grounding with calibrated decoding to reduce hallucinations, and introduces alignment-aware binarized aggregation to enhance robustness while maintaining bounded reconstruction quality. Across benchmarks on factuality, toxicity, and membership inference, SafeLM achieves 98.0% harmful content detection accuracy, reduces communication by 96.9%, and lowers gradient inversion PSNR from 31.7 dB to 15.1 dB. Ablations show that each component contributes independently, whereas their integration yields a strong privacy utility efficiency trade-off for deploying trustworthy LLMs.

Problem

• privacy
• security
• misinformation
• adversarial robustness
• federated learning

Innovation

• Federated Learning
• Privacy-Preserving LLMs
• Gradient Smartification
• Adversarial Robustness
• Contrastive Grounding