Federated Bilevel Performative Prediction

2026-06-17

AI Summary

This work addresses the challenge in federated bilevel optimization where client data distributions shift due to strategic decision-making—referred to as performativity—leading to model failure. It formally introduces the notion of a federated bilevel performative stable point for the first time, proposing a decoupled risk perspective to define stable solutions and establishing conditions for their existence and uniqueness. Methodologically, two algorithms with convergence guarantees are developed: one achieving linear convergence and the other prioritizing communication efficiency, both leveraging federated hypergradient estimation, two-timescale updates, and contraction mapping analysis. Experiments on strategic regression and meta-learning tasks validate the efficacy of the proposed stability threshold, demonstrating significant improvements over non-performative baselines in meta-generalization, while CNN-based classification tasks further confirm practical performance gains.

Abstract

Federated bilevel optimization is widely used for nested learning problems across distributed clients, such as federated hyperparameter tuning and meta-learning under privacy and communication constraints. Most existing formulations assume fixed client data distributions, which can be violated by performativity, where deployed decisions reshape client behavior and data collection, inducing client-specific, decision-dependent distribution shift. We study federated bilevel performative prediction, where both upper-level (UL) and lower-level (LL) objectives are evaluated under client-dependent, decision-dependent distributions. We formalize the federated bilevel performatively stable (FBPS) point under a decoupled-risk perspective and provide sufficient conditions for its existence and uniqueness. We then develop two federated methods to compute the FBPS solution: FBi-RRM, which converges linearly under a contraction condition, and FBi-SGD, a communication-efficient stochastic method based on federated hypergradient estimation with convergence guarantees under diminishing step sizes when sensitivities are sufficiently small. Experiments on strategic regression and meta strategic classification validate the predicted stability thresholds and demonstrate improved meta-generalization over non-performative baselines, and CNN-based classification further demonstrates the practical effectiveness of the proposed methods in nonconvex neural network settings.

Problem

• federated bilevel optimization
• performative prediction
• distribution shift
• decision-dependent distribution
• federated learning

Innovation

• federated bilevel optimization
• performative prediction
• decision-dependent distribution shift
• federated hypergradient estimation
• meta-generalization