CA-HFP: Curvature-Aware Heterogeneous Federated Pruning with Model Reconstruction

2026-03-12

AI Summary

This work addresses the challenge of personalized model compression in federated learning over heterogeneous edge devices by proposing a curvature-aware heterogeneous federated pruning framework. The method introduces loss curvature information into the pruning strategy for the first time, enabling each client to perform structured, device-adaptive pruning guided by curvature-based importance scores. A lightweight reconstruction step maps the pruned submodels back into a unified global parameter space, ensuring compatibility during aggregation and convergence of training. Theoretical analysis establishes convergence bounds under multi-step local SGD and derives a loss-based pruning criterion. Experiments on FMNIST, CIFAR-10, and CIFAR-100 demonstrate that the proposed approach significantly reduces communication and computational overhead while achieving higher accuracy than standard federated learning and existing pruning baselines.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Non-convex Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Federated learning on heterogeneous edge devices requires personalized compression while preserving aggregation compatibility and stable convergence. We present Curvature-Aware Heterogeneous Federated Pruning (CA-HFP), a practical framework that enables each client perform structured, device-specific pruning guided by a curvature-informed significance score, and subsequently maps its compact submodel back into a common global parameter space via a lightweight reconstruction. We derive a convergence bound for federated optimization with multiple local SGD steps that explicitly accounts for local computation, data heterogeneity, and pruning-induced perturbations; from which a principled loss-based pruning criterion is derived. Extensive experiments on FMNIST, CIFAR-10, and CIFAR-100 using VGG and ResNet architectures under varying degrees of data heterogeneity demonstrate that CA-HFP preserves model accuracy while significantly reducing per-client computation and communication costs, outperforming standard federated training and existing pruning-based baselines.

Problem

• federated learning
• model compression
• heterogeneous devices
• pruning
• convergence stability

Innovation

• federated pruning
• curvature-aware
• model reconstruction
• heterogeneous devices
• convergence analysis