Learning Confidence Ellipsoids and Applications to Robust Subspace Recovery

2025-12-18

AI Summary

This paper addresses the efficient learning of minimum-volume confidence ellipsoids for arbitrary high-dimensional distributions: given i.i.d. samples and a confidence level α, find the ellipsoid E of minimal volume satisfying Pr_D[E] ≥ 1−α. The problem is NP-hard when the ellipsoid’s condition number β is unbounded; thus, we focus on the β-bounded regime. We propose the first polynomial-time algorithm achieving an O(β^{γd}) volume approximation ratio—where γ > 0 is a small constant—and prove that this exponential dependence on d is computationally nearly tight. Our approach integrates the primal-dual structure of the minimum-volume enclosing ellipsoid problem with geometric Brascamp–Lieb inequalities to formulate a robust optimization framework. This yields the first polynomial-time robust subspace recovery algorithm with worst-case theoretical guarantees. The method significantly enhances stability and practicality in high-dimensional anomaly detection and dimensionality reduction.

Technology Category

• Machine Learning: Learning with Manifolds
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements

Abstract

We study the problem of finding confidence ellipsoids for an arbitrary distribution in high dimensions. Given samples from a distribution $D$ and a confidence parameter $alpha$, the goal is to find the smallest volume ellipsoid $E$ which has probability mass $Pr_{D}[E] ge 1-alpha$. Ellipsoids are a highly expressive class of confidence sets as they can capture correlations in the distribution, and can approximate any convex set. This problem has been studied in many different communities. In statistics, this is the classic minimum volume estimator introduced by Rousseeuw as a robust non-parametric estimator of location and scatter. However in high dimensions, it becomes NP-hard to obtain any non-trivial approximation factor in volume when the condition number $eta$ of the ellipsoid (ratio of the largest to the smallest axis length) goes to $infty$. This motivates the focus of our paper: can we efficiently find confidence ellipsoids with volume approximation guarantees when compared to ellipsoids of bounded condition number $eta$? Our main result is a polynomial time algorithm that finds an ellipsoid $E$ whose volume is within a $O(eta^{gamma d})$ multiplicative factor of the volume of best $eta$-conditioned ellipsoid while covering at least $1-O(alpha/gamma)$ probability mass for any $gamma<alpha$. We complement this with a computational hardness result that shows that such a dependence seems necessary up to constants in the exponent. The algorithm and analysis uses the rich primal-dual structure of the minimum volume enclosing ellipsoid and the geometric Brascamp-Lieb inequality. As a consequence, we obtain the first polynomial time algorithm with approximation guarantees on worst-case instances of the robust subspace recovery problem.

Problem

• Efficiently find confidence ellipsoids with volume approximation guarantees
• Address NP-hardness in high dimensions for arbitrary distributions
• Provide polynomial-time algorithm for robust subspace recovery problem

Innovation

• Polynomial time algorithm for confidence ellipsoids with volume guarantees.
• Uses primal-dual structure and geometric Brascamp-Lieb inequality.
• Provides first polynomial time approximation for robust subspace recovery.