HAL-MLE Log-Splines Density Estimation (Part I: Univariate)

2026-02-18

AI Summary

This study addresses efficient maximum likelihood estimation under total variation (TV) regularization in one-dimensional nonparametric density estimation. Building upon the highly adaptive Lasso (HAL) framework and employing a log-spline link function, the authors construct a maximum likelihood estimator that, for the first time in the univariate setting, establishes an equivalence between the sectional variation norm of HAL and the classical bounded TV assumption. Theoretical analysis demonstrates that the estimator is asymptotically linear and pointwise asymptotically normal. Moreover, when the underlying density possesses smoothness of order \(k \geq 1\), the estimator achieves a uniform convergence rate of \(n^{-(k+1)/(2k+3)}\) up to logarithmic factors—surpassing prior results that only guaranteed consistency—and thereby laying a rigorous theoretical foundation for future multidimensional extensions.

Technology Category

• Machine Learning: Learning Theory
• Reasoning under Uncertainty: Graphical Models
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We study nonparametric maximum likelihood estimation of probability densities under a total variation (TV) type penalty, sectional variation norm (also named as Hardy-Krause variation). TV regularization has a long history in regression and density estimation, including results on $L^2$ and KL divergence convergence rates. Here, we revisit this task using the Highly Adaptive Lasso (HAL) framework. We formulate a HAL-based maximum likelihood estimator (HAL-MLE) using the log-spline link function from \citet{kooperberg1992logspline}, and show that in the univariate setting the bounded sectional variation norm assumption underlying HAL coincides with the classical bounded TV assumption. This equivalence directly connects HAL-MLE to existing TV-penalized approaches such as local adaptive splines \citep{mammen1997locally}. We establish three new theoretical results: (i) the univariate HAL-MLE is asymptotically linear, (ii) it admits pointwise asymptotic normality, and (iii) it achieves uniform convergence at rate $n^{-(k+1)/(2k+3)}$ up to logarithmic factors for the smoothness order $k \geq 1$. These results extend existing results from \citet{van2017uniform}, which previously guaranteed only uniform consistency without rates when $k=0$. We will include the uniform convergence for general dimension $d$ in the follow-up work of this paper. The intention of this paper is to provide a unified framework for the TV-penalized density estimation methods, and to connect the HAL-MLE to the existing TV-penalized methods in the univariate case, despite that the general HAL-MLE is defined for multivariate cases.

Problem

• density estimation
• total variation penalty
• nonparametric maximum likelihood
• univariate
• sectional variation norm

Innovation

• HAL-MLE
• total variation regularization
• log-spline density estimation
• asymptotic normality
• uniform convergence rate