Local EGOP for Continuous Index Learning

2026-01-11

AI Summary

High-dimensional functions often exhibit local variation only along a few directions, posing challenges for traditional methods to efficiently capture their intrinsic structure. This work proposes the Local EGOP algorithm, which, within a continuous index learning framework, uniquely employs the Expected Gradient Outer Product (EGOP) both as a local metric and as an estimator of the inverse covariance of the target distribution, thereby adaptively identifying the function’s local active subspace. By integrating recursive kernel adaptation with nonparametric regression, the method achieves adaptive learning of the intrinsic dimensionality of functions embedded in arbitrary high-dimensional noise under the supervised noise manifold assumption. Experiments demonstrate that Local EGOP outperforms two-layer neural networks in regression tasks on continuous single-index models and exhibits feature learning capabilities comparable to those of deep learning approaches.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Local Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We introduce the setting of continuous index learning, in which a function of many variables varies only along a small number of directions at each point. For efficient estimation, it is beneficial for a learning algorithm to adapt, near each point $x$, to the subspace that captures the local variability of the function $f$. We pose this task as kernel adaptation along a manifold with noise, and introduce Local EGOP learning, a recursive algorithm that utilizes the Expected Gradient Outer Product (EGOP) quadratic form as both a metric and inverse-covariance of our target distribution. We prove that Local EGOP learning adapts to the regularity of the function of interest, showing that under a supervised noisy manifold hypothesis, intrinsic dimensional learning rates are achieved for arbitrarily high-dimensional noise. Empirically, we compare our algorithm to the feature learning capabilities of deep learning. Additionally, we demonstrate improved regression quality compared to two-layer neural networks in the continuous single-index setting.

Problem

• continuous index learning
• local subspace adaptation
• high-dimensional function estimation
• intrinsic dimensionality
• manifold learning

Innovation

• Local EGOP
• continuous index learning
• Expected Gradient Outer Product
• intrinsic dimensionality
• manifold learning