Nonconvex Penalized LAD Estimation in Partial Linear Models with DNNs: Asymptotic Analysis and Proximal Algorithms

2025-11-26

AI Summary

This paper addresses the statistical inference challenges arising from LAD estimation and nonconvex, nonsmooth regularization in high-dimensional partially linear models. Methodologically, it proposes a novel deep neural network (DNN)-based estimation framework—first embedding nonconvex penalties (e.g., SCAD, MCP) into DNN-driven partially linear models, coupled with least absolute deviations loss, and optimized efficiently via infinite-dimensional variational analysis and proximal subgradient algorithms. Theoretically, it establishes consistency, optimal convergence rates, and asymptotic normality of the estimators; proves convergence of the relaxed optimization problem; and characterizes the fundamental trade-off between computational efficiency and statistical accuracy induced by nonconvex regularization. By simultaneously accommodating model high-dimensionality, objective nonconvexity, loss nonsmoothness, and DNN architectural complexity, this work provides a unified framework for robust high-dimensional inference.

Technology Category

• Search and Optimization: Non-convex Optimization
• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

This paper investigates the partial linear model by Least Absolute Deviation (LAD) regression. We parameterize the nonparametric term using Deep Neural Networks (DNNs) and formulate a penalized LAD problem for estimation. Specifically, our model exhibits the following challenges. First, the regularization term can be nonconvex and nonsmooth, necessitating the introduction of infinite dimensional variational analysis and nonsmooth analysis into the asymptotic normality discussion. Second, our network must expand (in width, sparsity level and depth) as more samples are observed, thereby introducing additional difficulties for theoretical analysis. Third, the oracle of the proposed estimator is itself defined through a ultra high-dimensional, nonconvex, and discontinuous optimization problem, which already entails substantial computational and theoretical challenges. Under such the challenges, we establish the consistency, convergence rate, and asymptotic normality of the estimator. Furthermore, we analyze the oracle problem itself and its continuous relaxation. We study the convergence of a proximal subgradient method for both formulations, highlighting their structural differences lead to distinct computational subproblems along the iterations. In particular, the relaxed formulation admits significantly cheaper proximal updates, reflecting an inherent trade-off between statistical accuracy and computational tractability.

Problem

• Estimating partial linear models with nonconvex penalties using DNNs
• Addressing asymptotic normality under nonsmooth nonconvex regularization
• Solving ultra high-dimensional discontinuous optimization for LAD estimation

Innovation

• LAD regression with DNNs in partial linear models
• Nonconvex penalized estimation requiring asymptotic analysis
• Proximal algorithms for high-dimensional discontinuous optimization