Risk reversal for least squares estimators under nested convex constraints

2026-01-22

AI Summary

This study investigates the statistical risk behavior of projection-based least squares estimators under nested convex constraints in Gaussian sequence models. Employing tools from Euclidean projection, tangent cone analysis, and statistical dimension, the work systematically examines how noise magnitude influences estimation performance. Its central contribution is the first identification of a “risk reversal” phenomenon: under high noise levels, tighter convex constraints can strictly increase estimation risk. This counterintuitive effect is rigorously shown to arise from the global geometric structure of the constraint sets. The paper further constructs explicit counterexamples demonstrating that the worst-case risk over a smaller constraint set can significantly exceed that over a larger one, thereby challenging the conventional intuition that tighter constraints invariably yield better estimates.

Technology Category

• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

In constrained stochastic optimization, one naturally expects that imposing a stricter feasible set does not increase the statistical risk of an estimator defined by projection onto that set. In this paper, we show that this intuition can fail even in canonical settings. We study the Gaussian sequence model, a deliberately austere test best, where for a compact, convex set $\Theta \subset \mathbb{R}^d$ one observes \[ Y = \theta^\star + \sigma Z, \qquad Z \sim N(0, I_d), \] and seeks to estimate an unknown parameter $\theta^\star \in \Theta$. The natural estimator is the least squares estimator (LSE), which coincides with the Euclidean projection of $Y$ onto $\Theta$. We construct an explicit example exhibiting \emph{risk reversal}: for sufficiently large noise, there exist nested compact convex sets $\Theta_S \subset \Theta_L$ and a parameter $\theta^\star \in \Theta_S$ such that the LSE constrained to $\Theta_S$ has strictly larger risk than the LSE constrained to $\Theta_L$. We further show that this phenomenon can persist at the level of worst-case risk, with the supremum risk over the smaller constraint set exceeding that over the larger one. We clarify this behavior by contrasting noise regimes. In the vanishing-noise limit, the risk admits a first-order expansion governed by the statistical dimension of the tangent cone at $\theta^\star$, and tighter constraints uniformly reduce risk. In contrast, in the diverging-noise regime, the risk is determined by global geometric interactions between the constraint set and random noise directions. Here, the embedding of $\Theta_S$ within $\Theta_L$ can reverse the risk ordering. These results reveal a previously unrecognized failure mode of projection-based estimators: in sufficiently noisy settings, tightening a constraint can paradoxically degrade statistical performance.

Problem

• risk reversal
• least squares estimator
• convex constraints
• statistical risk
• Gaussian sequence model

Innovation

• risk reversal
• least squares estimator
• convex constraints
• statistical risk
• Gaussian sequence model