Efficient First-Order Methods for Estimating Generalized Additive Index Models

2026-05-27

AI Summary

This work addresses the computational inefficiency, reliance on nonparametric smoothing, and sequential nature of classical two-stage estimation procedures for generalized additive index models (GAIMs). The authors propose a simultaneous estimation algorithm that transforms the semiparametric problem into a finite-dimensional optimization via basis function expansions and solves it using first-order optimization methods. Innovatively extending the variational inequality (VI) framework from generalized linear models to GAIMs, the study provides the first unified convergence guarantee to stationary points for both gradient descent and VI algorithms. Numerical experiments demonstrate that the proposed method outperforms classical approaches in both computational efficiency and statistical performance, with the VI algorithm showing particular advantages under nonstandard link functions.

Abstract

Generalized additive index models (GAIMs) offer a flexible semiparametric framework for capturing complex data relationships, balancing the interpretability of parametric models with the flexibility of nonparametric approaches. However, classical stage-wise estimation procedures for GAIMs suffer from computational inefficiencies due to their sequential nature and reliance on nonparametric smoothing. To overcome these drawbacks, we propose efficient, simultaneous estimation algorithms for GAIMs. By leveraging basis expansion, we cast the semiparametric estimation task as a finite-dimensional optimization problem solvable by first-order methods such as gradient descent (GD). Furthermore, we introduce a variational inequality (VI) estimation algorithm, extending the VI framework from generalized linear models to GAIMs. We provide a unified convergence result to a stationary point for both algorithms. Numerical experiments highlight the computational and statistical advantages of our methods over classical stage-wise procedures, and reveal the potential benefits of the VI-based approach over GD for non-canonical link functions.

Problem

• Generalized Additive Index Models
• computational inefficiency
• stage-wise estimation
• nonparametric smoothing

Innovation

• generalized additive index models
• first-order methods
• basis expansion
• variational inequality
• simultaneous estimation