Low Rank Comes with Low Security: Gradient Assembly Poisoning Attacks against Distributed LoRA-based LLM Systems

2026-01-02

arXiv.org

AI Summary

This work identifies a critical structural vulnerability in Low-Rank Adaptation (LoRA) within federated learning settings, where the unverified product of client-submitted low-rank matrices \( A \) and \( B \) introduces a security blind spot. The authors propose Gradient Assembly Poisoning (GAP), a novel attack that requires neither access to training data nor cross-client coordination. By carefully crafting seemingly compliant \( A \) and \( B \) matrices whose product induces malicious effects, GAP stealthily degrades model performance while evading standard anomaly detection mechanisms. Extensive experiments on LLaMA, ChatGLM, and GPT-2 demonstrate the attack’s efficacy: BLEU scores drop by up to 14.5%, factual and grammatical errors surge by over 800%, and generated text retains 92.6% of its original length, highlighting the severity and subtlety of this threat.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Low-Rank Adaptation (LoRA) has become a popular solution for fine-tuning large language models (LLMs) in federated settings, dramatically reducing update costs by introducing trainable low-rank matrices. However, when integrated with frameworks like FedIT, LoRA introduces a critical vulnerability: clients submit $A$ and $B$ matrices separately, while only their product $AB$ determines the model update, yet this composite is never directly verified. We propose Gradient Assembly Poisoning (GAP), a novel attack that exploits this blind spot by crafting individually benign $A$ and $B$ matrices whose product yields malicious updates. GAP operates without access to training data or inter-client coordination and remains undetected by standard anomaly detectors. We identify four systemic vulnerabilities in LoRA-based federated systems and validate GAP across LLaMA, ChatGLM, and GPT-2. GAP consistently induces degraded or biased outputs while preserving surface fluency, reducing BLEU by up to 14.5\%, increasing factual and grammatical errors by over 800\%, and maintaining 92.6\% long-form response length. These results reveal a new class of stealthy, persistent threats in distributed LoRA fine-tuning.

Problem

• Low-Rank Adaptation
• Federated Learning
• Poisoning Attack
• Large Language Models
• Gradient Assembly

Innovation

• Gradient Assembly Poisoning
• LoRA
• federated learning
• model poisoning
• low-rank adaptation