Detecting Atypical Clients in Federated Learning via Representation-Level Divergence

2026-05-21

AI Summary

This work addresses the challenge in federated learning where client data heterogeneity and anomalous behaviors often lead to unstable model updates, complicating the distinction between benign distribution shifts and harmful outliers. To this end, the paper proposes a lightweight, permutation-invariant geometric divergence metric that leverages a shared probing set to analyze discrepancies in how local and global models partition the input space at the representation level. By focusing on functional behavior in the representation space rather than model parameters or gradients, the method accurately quantifies each client’s functional deviation and effectively discriminates between stably heterogeneous clients and truly anomalous ones, thereby providing a reliable basis for risk-aware aggregation.

Abstract

Federated learning enables collaborative training across distributed clients with heterogeneous data, but such heterogeneity often leads to unstable updates and degraded global performance. Moreover, in practical deployments, client updates may deviate from the expected behavior not only due to benign not i.i.d. distributions, but also due to distributional shifts or anomalous inputs, raising concerns about the reliability of the aggregation process. In this work, we propose a lightweight geometric signal to quantify the functional deviation of a client with respect to the global model. Instead of comparing model parameters or gradients, our approach measures how the local training of each client alters the activation-induced partition of the input space, evaluated on a shared probe set. This yields a permutation-invariant, interpretable metric of client--global divergence that captures differences in how data is processed by the model. We show that this signal effectively identifies clients that induce atypical functional changes, distinguishing stable yet heterogeneous clients from those whose updates significantly diverge from the global regime. As a result, the proposed metric provides a simple tool for monitoring client behavior and enabling risk-aware aggregation strategies in federated learning systems.

Problem

• Federated Learning
• Client Heterogeneity
• Atypical Clients
• Distributional Shift
• Anomalous Inputs

Innovation

• representation-level divergence
• federated learning
• atypical client detection
• activation-induced partition
• risk-aware aggregation