🤖 AI Summary
This work addresses the high computational overhead of existing Byzantine-robust aggregation methods, which hinders their deployment in resource-constrained, large-scale federated learning systems. To overcome this limitation, the authors propose a lightweight Byzantine detection mechanism termed “Statistical Handcuffs,” which operates within the standard FedAvg framework. By extracting low-dimensional statistical fingerprints—such as update norms, inter-layer ratios, sparsity, and low-order moments—from client updates, the method constructs a detection space without modifying the underlying optimization process. This design ensures that adversaries cannot simultaneously evade detection and effectively poison the model. The approach is architecture-agnostic and particularly well-suited for LoRA-based federated fine-tuning. Experimental results demonstrate that under extreme non-IID settings with 50–150 clients and various attack types, the model maintains over 95% accuracy while achieving a stable detection precision of 0.8.
📝 Abstract
Existing Byzantine robust aggregation mechanisms typically rely on fulldimensional gradi ent comparisons or pairwise distance computations, resulting in computational overhead that limits applicability in large scale and resource constrained federated systems. This paper proposes TinyGuard, a lightweight Byzantine defense that augments the standard FedAvg algorithm via statistical update f ingerprinting. Instead of operating directly on high-dimensional gradients, TinyGuard extracts compact statistical fingerprints cap turing key behavioral properties of client updates, including norm statistics, layer-wise ratios, sparsity measures, and low-order mo ments. Byzantine clients are identified by measuring robust sta tistical deviations in this low-dimensional fingerprint space with nd complexity, without modifying the underlying optimization procedure. Extensive experiments on MNIST, Fashion-MNIST, ViT-Lite, and ViT-Small with LoRA adapters demonstrate that TinyGuard pre serves FedAvg convergence in benign settings and achieves up to 95 percent accuracy under multiple Byzantine attack scenarios, including sign-flipping, scaling, noise injection, and label poisoning. Against adaptive white-box adversaries, Pareto frontier analysis across four orders of magnitude confirms that attackers cannot simultaneously evade detection and achieve effective poisoning, features we term statistical handcuffs. Ablation studies validate stable detection precision 0.8 across varying client counts (50-150), threshold parameters and extreme data heterogeneity . The proposed framework is architecture-agnostic and well-suited for federated fine-tuning of foundation models where traditional Byzantine defenses become impractical