Locally Near Optimal Piecewise Linear Regression in High Dimensions via Difference of Max-Affine Functions

2026-05-07

AI Summary

This work addresses the challenge of efficient modeling and estimation in high-dimensional piecewise linear regression by introducing a novel parameterization framework based on the Difference of Maximum Affine functions (DoMA) and proposing an Adaptive Block Gradient Descent (ABGD) algorithm for optimization. Under sub-Gaussian noise assumptions, the method achieves, for the first time, local linear convergence and minimax-optimal sample complexity: in the noiseless setting, it exactly recovers the model with only Õ(d) samples, while in the presence of noise, it attains ε-accurate estimation using Õ(d(σ_z/ε)²) samples. The theoretical analysis provides non-asymptotic guarantees encompassing both local convergence and statistical learning performance. Empirical evaluations demonstrate that the proposed approach significantly outperforms existing state-of-the-art methods on real-world datasets.

Abstract

This paper presents a parametric solution to piecewise linear regression through the Adaptive Block Gradient Descent (ABGD) algorithm. The heart of the method is the parametrization of piecewise linear functions as the difference of max-affine (DoMA) functions. A non-asymptotic local convergence analysis for ABGD is provided under sub-Gaussian covariate and noise distributions. To initialize ABGD, we adapt a prior algorithm originally developed for the simpler setting of max-affine functions. When suitably initialized, ABGD converges linearly to an $ε$-accurate estimate given $\tilde{\mathcal{O}}(d\max(σ_z/ε,1)^2)$ observations where $σ_z^2$ denotes the noise variance. This implies exact recovery given $\tilde{\mathcal{O}}(d)$ samples in the noiseless case. Also, such a rate is shown to be minimax optimal up to logarithmic factors. Synthetic numerical results corroborate the theoretical guarantees for ABGD. We also observe competitive performance compared to the state-of-the-art methods on real-world datasets.

Problem

• piecewise linear regression
• high dimensions
• local optimality
• max-affine functions
• non-asymptotic convergence

Innovation

• piecewise linear regression
• difference of max-affine functions
• adaptive block gradient descent
• non-asymptotic convergence
• minimax optimality