FedBCGD: Communication-Efficient Accelerated Block Coordinate Gradient Descent for Federated Learning
This work addresses the high communication overhead of large-scale models, such as Vision Transformers, in federated learning by proposing Federated Block Coordinate Gradient Descent (FedBCGD) and its accelerated variant, FedBCGD+. The method introduces, for the first time in federated learning, a block-wise parameter communication mechanism that uploads only a subset of parameter blocks per round, combined with stochastic variance reduction and client drift control strategies. Theoretical analysis shows that the communication complexity is reduced by a factor of 1/N compared to existing methods, where N denotes the number of blocks. Experimental results demonstrate that the proposed algorithms achieve faster convergence and higher communication efficiency than current state-of-the-art approaches.