Laplace Approximations for Mixed-Effects and Gaussian Process Quantile Regression

2026-05-20

AI Summary

This work addresses the failure of the classical Laplace approximation in quantile regression with non-smooth, asymmetric Laplace likelihoods, where the observed Hessian is almost everywhere zero. The authors propose a novel Laplace approximation framework tailored for mixed-effects and Gaussian process quantile regression that avoids smoothing the likelihood. Instead, it constructs the approximation using either the Fisher information under correct model specification or the expected loss curvature under misspecification. Practical curvature estimators—such as the triangular kernel curvature (TKC)—are introduced to extend Laplace-type approximations to non-smooth generalized posteriors. The resulting method exhibits strong scalability and numerical stability, achieving accuracy comparable to or better than Markov chain Monte Carlo and variational inference at substantially lower computational cost.

Abstract

Laplace approximations are a standard tool for computationally efficient inference in latent Gaussian models, but they fail for quantile regression with the asymmetric Laplace likelihood because the observed Hessian vanishes almost everywhere. We show that this obstacle can be overcome without smoothing the likelihood: the relevant local curvature is given not by the observed Hessian, but by the Fisher information when the model is correctly specified and by the population curvature of the expected loss under misspecification. On this basis, we develop a Laplace approximation framework for quantile regression with mixed-effects and Gaussian process models. We propose practical curvature estimators, including the triangular kernel curvature (TKC) estimator, that yield approximations for posterior distributions and marginal likelihoods, and we establish their asymptotic validity. Empirically, the proposed methods are scalable and numerically stable, and for latent Gaussian models, they achieve accuracy comparable to or better than MCMC and variational competitors at substantially lower computational costs. More broadly, the framework clarifies how Laplace approximations can be justified for non-smooth generalized posteriors through local quadratic behavior of the expected loss.

Problem

• Laplace approximation
• quantile regression
• mixed-effects models
• Gaussian processes
• non-smooth likelihood

Innovation

• Laplace approximation
• quantile regression
• Gaussian process
• mixed-effects models
• non-smooth likelihood