Zero-Inflated Gaussian Distributions Enable Parameter-Space Sparsity in Estimation-of-Distribution Algorithms

2026-06-11

AI Summary

This work addresses the challenge that existing Estimation of Distribution Algorithms (EDAs) struggle to effectively handle sparsity in parameter spaces, while conventional sparse black-box optimization methods rely on handcrafted sparsity-inducing operations that contradict EDAs’ goal of minimizing human bias. To overcome this, the study introduces, for the first time, a multivariate Zero-Inflated Gaussian (ZIG) distribution into the EDA framework. By employing a latent variable model that jointly captures both the zero/non-zero indicators and the non-zero magnitudes of parameters, the proposed ZIG-EDA enables end-to-end joint optimization of sparsity structure and active parameters. The method requires no predefined sparsity strategy, is identifiable under observed samples, and leverages an amortized inversion estimator to efficiently recover parameter dependencies. On the Lunar Lander benchmark, ZIG-EDA significantly outperforms dense Gaussian EDAs, hand-designed sparse evolutionary algorithms, and ad hoc sparse EDAs, achieving faster convergence, higher returns, and policies that activate only a minimal subset of parameters.

Abstract

Estimation-of-distribution algorithms (EDAs) are a powerful class of evolutionary methods for black-box optimization, especially when little is known about the structure of the objective. Whereas classical evolutionary algorithms rely on hand-designed mutation and crossover operators, hard to devise for unknown problem structures, and a source of bias, EDAs sidestep operator design entirely: they fit a probability distribution to the best individuals and sample the next generation from it. EDAs are well established on continuous parameter spaces, but they have not previously been generalized to sparse ones, in which most coefficients of a good solution are exactly zero. Existing sparse black-box optimizers therefore reintroduce exactly what EDAs were designed to avoid: hand-crafted sparsity operators, bi-level schemes alternating between support set and active values, zeroing thresholds, and other baked-in assumptions. We close this gap by proposing multivariate zero-inflated Gaussian (ZIG) distributions as EDA sampling laws. A latent Gaussian model with separate indicator and value dimensions represents sparsity patterns, correlations among active parameters, and the interactions between the two, so sparsity patterns and active values are optimized jointly, hierarchy-free. We show that the latent parameters of this model are identifiable from observed samples, unlike in the missing-data settings where related constructions originate, and introduce practical amortized inversion-based estimators for them. The estimators accurately recover latent correlation structures, and on the Lunar Lander benchmark the resulting ZIG-EDA converges faster and reaches higher final returns than a dense Gaussian EDA, a hand-crafted sparse evolutionary algorithm, and an ad-hoc sparse EDA, while finding controllers with only a small fraction of parameters active.

Problem

• sparsity
• estimation-of-distribution algorithms
• black-box optimization
• parameter-space sparsity
• zero-inflated distributions

Innovation

• Zero-Inflated Gaussian
• Estimation-of-Distribution Algorithms
• Sparsity
• Latent Variable Model
• Black-Box Optimization