Distribution-Aware Robust Bilevel Optimization: Quantile-Guided Huber Updates in Two-Timescale Stochastic Approximation

2026-06-21

AI Summary

This work addresses the vulnerability of bilevel optimization to outlier gradients under heavy-tailed noise, where existing variance-reduction techniques struggle to distinguish informative signals from impulsive noise. To overcome this limitation, we propose the RQ-TTSA framework, which introduces a distribution-aware mechanism into bilevel optimization for the first time. By maintaining a historical gradient buffer to estimate rolling quantiles, RQ-TTSA adaptively adjusts the Huber clipping threshold, thereby tightly controlling effective variance while preserving local geometric structure. The method establishes convergence guarantees for nonconvex–strongly-convex bilevel problems even under infinite-variance noise. Empirically, it consistently outperforms baselines across six benchmark tasks—eliminating divergence spikes, enabling stable convergence, exhibiting robustness to hyperparameter variations, and incurring only a modest 2.7% increase in computational overhead.

Abstract

Bilevel optimization (BLO) is fundamental to hierarchical decision-making but suffers from critical instability under heavy-tailed stochastic noise. Existing variance-reduction techniques typically rely on myopic magnitude checks, which fail to distinguish informative geometric signals from impulsive outliers. To resolve this, we propose \textbf{RQ-TTSA} (Robust Quantile-guided TTSA), a distribution-aware framework that leverages historical gradient buffers to estimate rolling quantiles for adaptive Huber-style clipping, effectively preserving local optimization geometry while strictly bounding effective variance. Theoretically, we provide a convergence analysis for quantile-guided TTSA under nonconvex-strongly convex assumptions with infinite-variance noise ($p \in (1,2]$), deriving a rate of $\mathcal{O}(T^{-\frac{p-1}{3p-2}})$ that recovers optimal dependence on the heavy-tailed parameter. Empirically, across six diverse tasks, spanning heterogeneous vision benchmarks, dynamic games under momentum poisoning, and offline reinforcement learning, RQ-TTSA consistently outperforms state-of-the-art baselines by eliminating divergence spikes and ensuring stable convergence. Our method demonstrates significant robustness to hyperparameter variations and incurs negligible computational overhead ($\approx 2.7\%$ increase), validating distribution-aware gradient control as a practical and necessary component for reliable bilevel learning.

Problem

• Bilevel Optimization
• Heavy-tailed Noise
• Robustness
• Stochastic Approximation
• Outlier

Innovation

• distribution-aware optimization
• quantile-guided clipping
• robust bilevel optimization
• heavy-tailed noise
• two-timescale stochastic approximation