FedTVD: Balancing Data Quality and Quantity for Robust Federated Learning

2026-08-10

AI Summary

This work addresses the challenges of model bias and convergence difficulties in federated learning caused by skewed label distributions and heterogeneous data volumes across clients. To mitigate these issues, the authors propose FedTVD, a novel algorithm that introduces the Total Variation Distance (TVD) to quantify the divergence between each client’s label distribution and the global uniform distribution. FedTVD employs a dual-weighted aggregation mechanism that jointly accounts for both data quality (as measured by TVD) and local data quantity, effectively alleviating the adverse impact of non-IID data while remaining compatible with existing federated learning frameworks. Experimental results demonstrate that FedTVD consistently outperforms baseline methods across FMNIST, CIFAR-10, and CIFAR-100 benchmarks, achieving a 10.6% accuracy improvement over FedAvg under highly skewed label distributions on CIFAR-10, while maintaining superior performance in moderately skewed and IID settings.

Abstract

Federated Learning (FL) enables collaborative model training across distributed client devices while preserving data privacy. However, FL faces significant challenges due to data heterogeneity, particularly in terms of label distribution skewness and variations in dataset sizes, which can lead to biased model updates and hinder convergence. To address this, we propose FedTVD, a novel FL algorithm that weights client contributions during aggregation by considering both data quality and quantity. Unlike traditional FL approaches such as FedAvg, which rely solely on dataset size for client weighting, FedTVD integrates Total Variation Distance (TVD) to measure the divergence between each client's local label distribution and a uniform global distribution. Clients with highly skewed distributions receive lower weights, preventing unbalanced datasets with imbalances from disproportionately influencing the global model. At the same time, dataset size is incorporated to ensure scalability and fairness. This dual-weighting mechanism effectively mitigates the impact of data imbalance, leading to more stable and generalized global models. Experimental results show that FedTVD consistently outperforms state-of-the-art methods across all datasets (FMNIST, CIFAR-10, and CIFAR-100) and all levels of data heterogeneity. Notably, it achieves up to 10.6% improvement over FedAvg on CIFAR-10 under highly skewed data, while maintaining top performance even under moderate and IID settings.

Problem

• Federated Learning
• Data Heterogeneity
• Label Distribution Skewness
• Dataset Size Imbalance
• Model Convergence

Innovation

• Federated Learning
• Total Variation Distance
• Data Heterogeneity
• Client Weighting
• Label Distribution Skew