Statistical-Computational Trade-offs in Learning Multi-Index Models via Harmonic Analysis

2026-02-10

AI Summary

This work investigates the learning problem for multi-index models (MIMs), where labels depend only on the projection of inputs onto an unknown low-dimensional subspace. Under the assumption of spherically symmetric input distributions, we provide the first sharp characterization via spherical harmonic analysis, leveraging representation theory of the orthogonal group and trace-zero symmetric tensors to establish fundamental computational–statistical lower bounds in both the statistical query and low-degree polynomial frameworks. Building on harmonic tensor expansions, we design a family of spectral algorithms that sequentially recover the latent directions and, under various choices of harmonic degrees, nearly achieve the theoretical lower bounds, thereby enabling a flexible trade-off between sample efficiency and runtime.

Technology Category

• Machine Learning: Matrix & Tensor Methods
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Graphical Models

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding

Abstract

We study the problem of learning multi-index models (MIMs), where the label depends on the input $\boldsymbol{x} \in \mathbb{R}^d$ only through an unknown $\mathsf{s}$-dimensional projection $\boldsymbol{W}_*^\mathsf{T} \boldsymbol{x} \in \mathbb{R}^\mathsf{s}$. Exploiting the equivariance of this problem under the orthogonal group $\mathcal{O}_d$, we obtain a sharp harmonic-analytic characterization of the learning complexity for MIMs with spherically symmetric inputs -- which refines and generalizes previous Gaussian-specific analyses. Specifically, we derive statistical and computational complexity lower bounds within the Statistical Query (SQ) and Low-Degree Polynomial (LDP) frameworks. These bounds decompose naturally across spherical harmonic subspaces. Guided by this decomposition, we construct a family of spectral algorithms based on harmonic tensor unfolding that sequentially recover the latent directions and (nearly) achieve these SQ and LDP lower bounds. Depending on the choice of harmonic degree sequence, these estimators can realize a broad range of trade-offs between sample and runtime complexity. From a technical standpoint, our results build on the semisimple decomposition of the $\mathcal{O}_d$-action on $L^2 (\mathbb{S}^{d-1})$ and the intertwining isomorphism between spherical harmonics and traceless symmetric tensors.

Problem

• multi-index models
• statistical-computational trade-offs
• harmonic analysis
• spherical symmetry
• learning complexity

Innovation

• multi-index models
• harmonic analysis
• statistical-computational trade-offs
• spherical harmonics
• spectral algorithms