Efficient sampling from a multivariate normal distribution subject to linear equality and inequality constraints

2025-08-21

AI Summary

This paper addresses efficient sampling from a multivariate normal distribution subject to joint linear equality and inequality constraints. Methodologically, it orthogonally decomposes the constrained space into the equality-constrained manifold and an inequality-constrained subspace; equality constraints are satisfied via a linear projection, while inequality constraints are handled using elliptical slice sampling, with an initial feasible point rapidly obtained via linear programming. This framework is the first to jointly model both constraint types without rejection, eliminating the inefficiency and numerical instability of traditional rejection sampling—especially in low-probability regions. Experiments in a four-dimensional constrained space (2 equality and 5 inequality constraints) demonstrate that the generated samples accurately recover the theoretical mean and covariance, with estimation errors under 1%, and zero sample rejections.

Technology Category

• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Large-scale security measurements
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Sampling from multivariate normal distributions, subjected to a variety of restrictions, is a problem that is recurrent in statistics and computing. In the present work, we demonstrate a general framework to efficiently sample a multivariate normal distribution subject to any set of linear inequality constraints and/or linear equality constraints simultaneously. In the approach we detail, sampling a multivariate random variable from the domain formed by the intersection of linear constraints proceeds via a combination of elliptical slice sampling to address the inequality constraints, and linear mapping to address the equality constraints. We also detail a linear programming method for finding an initial sample on the linearly constrained domain; such a method is critical for sampling problems where the domain has small probability. We demonstrate the validity of our methods on an arbitrarily chosen four-dimensional multivariate normal distribution subject to five inequality constraints and/or two equality constraints. Our approach compares favourably to direct sampling and/or accept-reject sampling methods; the latter methods vary widely in their efficiency, whereas the methods in the present work are rejection-free. Where practical we compare predictions of probability density functions between our sampling methods and analytical computation. For all simulations we demonstrate that our methods yield accurate computation of the mean and covariance of the multivariate normal distributions restricted by the imposed linear constraints. MATLAB codes to implement our methods are readily available at https://dx.doi.org/10.6084/m9.figshare.29956304 .

Problem

• Efficiently sampling multivariate normal distributions under linear constraints
• Combining elliptical slice sampling with linear mapping for constraints
• Finding initial samples in low-probability constrained domains via linear programming

Innovation

• Elliptical slice sampling for inequality constraints
• Linear mapping method for equality constraints
• Linear programming for initial sample generation