Runtime Analysis of the Compact Genetic Algorithm on the LeadingOnes Benchmark

2025-01-27

AI Summary

This work provides the first rigorous theoretical analysis of the compact genetic algorithm (cGA) on the LeadingOnes benchmark function, addressing a long-standing gap in its mathematical convergence guarantees. Using drift analysis, stochastic process modeling, and probabilistic techniques, we prove that when the population size satisfies $K = Omega( ext{polylog},n)$, cGA converges to the optimum with high probability within $O(n log n cdot K)$ function evaluations; under optimal parameter settings, this bound tightens to $O(n^2)$, up to polylogarithmic factors. This establishes the first quasi-linear convergence bound for cGA on LeadingOnes. Moreover, our analysis uncovers fundamental differences between cGA and other estimation-of-distribution algorithms (EDAs), such as UMDA, particularly in selection mechanisms and probabilistic model updates—moving beyond prior empirical or simplifying-assumption-based studies.

Technology Category

• Search and Optimization: Evolutionary Computation
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Other Foundations of Constraint Satisfaction

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Models for Web evolution
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

The compact genetic algorithm (cGA) is one of the simplest estimation-of-distribution algorithms (EDAs). Next to the univariate marginal distribution algorithm (UMDA) -- another simple EDA -- , the cGA has been subject to extensive mathematical runtime analyses, often showcasing a similar or even superior performance to competing approaches. Surprisingly though, up to date and in contrast to the UMDA and many other heuristics, we lack a rigorous runtime analysis of the cGA on the LeadingOnes benchmark -- one of the most studied theory benchmarks in the domain of evolutionary computation. We fill this gap in the literature by conducting a formal runtime analysis of the cGA on LeadingOnes. For the cGA's single parameter -- called the hypothetical population size -- at least polylogarithmically larger than the problem size, we prove that the cGA samples the optimum of LeadingOnes with high probability within a number of function evaluations quasi-linear in the problem size and linear in the hypothetical population size. For the best hypothetical population size, our result matches, up to polylogarithmic factors, the typical quadratic runtime that many randomized search heuristics exhibit on LeadingOnes. Our analysis exhibits some noteworthy differences in the working principles of the two algorithms which were not visible in previous works.

Problem

• Compact Genetic Algorithm
• LeadingOnes Problem
• Runtime Analysis

Innovation

• Compact Genetic Algorithm
• LeadingOnes Benchmark
• Mathematical Analysis