Influence Diagnostics in High-dimensional M-estimation: Precise Asymptotics

2026-07-10

AI Summary

This work addresses the lack of precise characterization of individual sample influence in high-dimensional convex M-estimation when the sample size and dimensionality are of the same order (\(n \sim d\)). Under Gaussian design assumptions, the authors rigorously derive the asymptotic distribution of influence measures by integrating high-dimensional asymptotic analysis, random matrix theory, and leave-one-out influence diagnostics. They establish that the influence measure converges to an explicitly expressible limiting distribution. The analysis further reveals that highly influential samples concentrate near the decision boundary, thereby providing theoretical justification for boundary-based heuristic strategies in active learning and uncovering an intrinsic connection between influence analysis and classification boundaries.

Abstract

The impact of a given training point on a statistical model is classically measured through its leave-one-out influence, which quantifies the effect of its removal from the training set on the model accuracy. While the statistics of leave-one-out influences are well understood in the low-dimensional, large sample limit $n\to \infty, d=O(1)$, they become more intricate in high dimensions, as the influence of a given sample develops non-trivial dependencies on all other training samples. For convex M-estimation under Gaussian design, in the high-dimensional limit $n\asymp d$, we show that the distribution of the influences across the training set converges to a limiting measure which we sharply characterize. Building on these results, we provide evidence that influential samples tend to lie close to the decision boundary, thereby making contact with a standard data selection heuristic in active learning.

Problem

• influence diagnostics
• high-dimensional statistics
• M-estimation
• leave-one-out influence
• asymptotic analysis

Innovation

• high-dimensional M-estimation
• leave-one-out influence
• precise asymptotics
• influential samples
• decision boundary