Single-Round Scalable Analytic Federated Learning

2025-12-03

AI Summary

Federated learning (FL) faces two key challenges: high communication overhead and performance degradation under non-independent and identically distributed (Non-IID) data. While Analytical Federated Learning (AFL) achieves single-round aggregation and distributional robustness, it is restricted to linear models; nonlinear extensions (e.g., DeepAFL) forfeit the single-round advantage. This paper proposes SAFLe, a framework enabling scalable nonlinear modeling under strict single-round communication constraints. Its core innovation lies in a bucketed feature representation coupled with sparse grouped embeddings—mathematically proven to be equivalent to high-dimensional linear regression. This equivalence preserves AFL’s single-aggregation mechanism while endowing it with strong nonlinear expressivity. Experiments across multiple benchmarks demonstrate that SAFLe significantly outperforms linear AFL and multi-round DeepAFL in accuracy, while maintaining minimal communication cost and excellent scalability—establishing a new state-of-the-art for analytical federated learning.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Natural Language Processing: Safety and Robustness
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs

Abstract

Federated Learning (FL) is plagued by two key challenges: high communication overhead and performance collapse on heterogeneous (non-IID) data. Analytic FL (AFL) provides a single-round, data distribution invariant solution, but is limited to linear models. Subsequent non-linear approaches, like DeepAFL, regain accuracy but sacrifice the single-round benefit. In this work, we break this trade-off. We propose SAFLe, a framework that achieves scalable non-linear expressivity by introducing a structured head of bucketed features and sparse, grouped embeddings. We prove this non-linear architecture is mathematically equivalent to a high-dimensional linear regression. This key equivalence allows SAFLe to be solved with AFL's single-shot, invariant aggregation law. Empirically, SAFLe establishes a new state-of-the-art for analytic FL, significantly outperforming both linear AFL and multi-round DeepAFL in accuracy across all benchmarks, demonstrating a highly efficient and scalable solution for federated vision.

Problem

• Achieves scalable non-linear expressivity in single-round federated learning
• Solves communication overhead and performance collapse on heterogeneous data
• Proves non-linear architecture equivalence to high-dimensional linear regression

Innovation

• Introduces structured head with bucketed features
• Uses sparse grouped embeddings for non-linear expressivity
• Mathematically equivalent to high-dimensional linear regression