FedCova: Robust Federated Covariance Learning Against Noisy Labels

2026-03-04

AI Summary

This work addresses the challenge of distributed label noise in federated learning, which often induces local overfitting and degrades global model performance. To mitigate this issue, the authors propose FedCova, a novel framework that uniquely leverages feature covariance as a unified mechanism for robust feature encoding, classifier construction, and label correction—without requiring external clean data or client filtering. FedCova formulates a federated lossy feature encoding objective via mutual information maximization and enhances classification through a subspace-aware classifier built upon class-conditional covariance and an error-tolerant term. Extensive experiments on benchmarks including CIFAR-10/100 and Clothing1M under symmetric, asymmetric, and heterogeneous noise settings demonstrate that FedCova significantly outperforms current state-of-the-art methods.

Technology Category

• Machine Learning: Distributed Machine Learning & Federated Learning
• Computer Vision: Adversarial Attacks & Robustness
• Constraint Satisfaction and Optimization: Distributed CSP/Optimization

Application Category

• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Noisy labels in distributed datasets induce severe local overfitting and consequently compromise the global model in federated learning (FL). Most existing solutions rely on selecting clean devices or aligning with public clean datasets, rather than endowing the model itself with robustness. In this paper, we propose FedCova, a dependency-free federated covariance learning framework that eliminates such external reliances by enhancing the model's intrinsic robustness via a new perspective on feature covariances. Specifically, FedCova encodes data into a discriminative but resilient feature space to tolerate label noise. Built on mutual information maximization, we design a novel objective for federated lossy feature encoding that relies solely on class feature covariances with an error tolerance term. Leveraging feature subspaces characterized by covariances, we construct a subspace-augmented federated classifier. FedCova unifies three key processes through the covariance: (1) training the network for feature encoding, (2) constructing a classifier directly from the learned features, and (3) correcting noisy labels based on feature subspaces. We implement FedCova across both symmetric and asymmetric noisy settings under heterogeneous data distribution. Experimental results on CIFAR-10/100 and real-world noisy dataset Clothing1M demonstrate the superior robustness of FedCova compared with the state-of-the-art methods.

Problem

• federated learning
• noisy labels
• covariance learning
• robustness
• label noise

Innovation

• federated learning
• covariance learning
• label noise robustness
• feature subspace
• mutual information maximization