SGFusion: Stochastic Geographic Gradient Fusion in Federated Learning

2025-10-27

AI Summary

To address weak model generalization in federated learning caused by geographical heterogeneity among mobile users, this paper proposes a geography-aware gradient fusion framework. Methodologically, it constructs a hierarchical random graph to model inter-regional data correlations, and designs a probabilistic sampling–driven geographic gradient fusion mechanism that leverages self-attention weights to enable efficient knowledge sharing among spatially similar regions; theoretical analysis establishes an upper bound on the model’s convergence error. Experiments on heart rate prediction data from six countries demonstrate significant improvements in regional model accuracy and adaptability, without increasing communication or computational overhead—thus balancing performance and scalability. The core innovation lies in explicitly embedding geographical spatial structure into the federated aggregation process, achieving— for the first time—the integration of Markov Chain Monte Carlo sampling with geography-aware gradient fusion.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper proposes Stochastic Geographic Gradient Fusion (SGFusion), a novel training algorithm to leverage the geographic information of mobile users in Federated Learning (FL). SGFusion maps the data collected by mobile devices onto geographical zones and trains one FL model per zone, which adapts well to the data and behaviors of users in that zone. SGFusion models the local data-based correlation among geographical zones as a hierarchical random graph (HRG) optimized by Markov Chain Monte Carlo sampling. At each training step, every zone fuses its local gradient with gradients derived from a small set of other zones sampled from the HRG. This approach enables knowledge fusion and sharing among geographical zones in a probabilistic and stochastic gradient fusion process with self-attention weights, such that "more similar" zones have "higher probabilities" of sharing gradients with "larger attention weights." SGFusion remarkably improves model utility without introducing undue computational cost. Extensive theoretical and empirical results using a heart-rate prediction dataset collected across 6 countries show that models trained with SGFusion converge with upper-bounded expected errors and significantly improve utility in all countries compared to existing approaches without notable cost in system scalability.

Problem

• Leveraging geographic data from mobile users in Federated Learning
• Modeling correlations among geographic zones using hierarchical random graphs
• Enabling knowledge sharing between zones via stochastic gradient fusion

Innovation

• Trains separate FL models per geographic zone
• Models zone correlations via hierarchical random graph
• Fuses gradients probabilistically with self-attention weights