Median-of-Means as an Extremal Convex Estimator and a Nonconvex Route to the Trimmed Oracle

2026-09-01

AI Summary

研究通过引入非凸块-Lp估计器，解决了在重尾和对抗性污染数据中鲁棒学习的问题，逼近了修剪块oracle常数。

Abstract

We revisit median-of-means estimation from a deterministic optimization viewpoint and develop a family of block-Lp estimators for robust learning with heavy-tailed and adversarially corrupted data. In a block contamination model with at least a fraction 1 minus epsilon of good blocks, we first show that every convex block M-estimator has worst-case robustness constant at least 1 divided by 1 minus 2 epsilon. This matches the classical median-of-means bound and proves that the trimmed-block oracle constant 1 divided by 1 minus epsilon cannot be attained within the convex class. We then introduce a nonconvex block-Lp family for p between 0 and 1 and derive finite-sample deterministic robustness bounds for all global minimizers. As p decreases from 1 toward 0, these bounds continuously approach the trimmed-block oracle constant. For sufficiently small p, the global minimizers coincide with those of the oracle under a mild separation condition. We also show that the block-Lp objectives have a benign landscape, with all local minima remaining close to the truth and no bad basins. Combining these results with block-level concentration yields sub-Gaussian deviation bounds under finite 2 plus delta moments and high-dimensional extensions to robust mean estimation and sparse regression.

Problem

• Median-of-Means
• Robust Learning
• Heavy-tailed Data
• Adversarial Corruption

Innovation

• block-Lp estimators
• robust learning
• nonconvex optimization
• trimmed-block oracle
• heavy-tailed data