Regime-Adaptive Bayesian Optimization via Dirichlet Process Mixtures of Gaussian Processes

2026-01-27

AI Summary

Standard Bayesian optimization struggles in multi-region problems—such as molecular conformation search and drug discovery—due to its assumption of global smoothness. This work proposes RAMBO, the first approach to integrate a Dirichlet process mixture of Gaussian processes into Bayesian optimization, enabling adaptive identification of latent regions without pre-specifying their number and constructing independent Gaussian process models for each region. Leveraging collapsed Gibbs sampling for efficient inference, RAMBO introduces a novel acquisition function that decomposes uncertainty into within-region and between-region components and employs an adaptive concentration parameter schedule to enable coarse-to-fine region exploration. Experiments on synthetic benchmarks and real-world tasks—including molecular conformation optimization, virtual drug screening, and fusion reactor design—demonstrate that RAMBO significantly outperforms state-of-the-art methods.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Intelligent Robots: Learning & Optimization for ROB
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Standard Bayesian Optimization (BO) assumes uniform smoothness across the search space an assumption violated in multi-regime problems such as molecular conformation search through distinct energy basins or drug discovery across heterogeneous molecular scaffolds. A single GP either oversmooths sharp transitions or hallucinates noise in smooth regions, yielding miscalibrated uncertainty. We propose RAMBO, a Dirichlet Process Mixture of Gaussian Processes that automatically discovers latent regimes during optimization, each modeled by an independent GP with locally-optimized hyperparameters. We derive collapsed Gibbs sampling that analytically marginalizes latent functions for efficient inference, and introduce adaptive concentration parameter scheduling for coarse-to-fine regime discovery. Our acquisition functions decompose uncertainty into intra-regime and inter-regime components. Experiments on synthetic benchmarks and real-world applications, including molecular conformer optimization, virtual screening for drug discovery, and fusion reactor design, demonstrate consistent improvements over state-of-the-art baselines on multi-regime objectives.

Problem

• Bayesian Optimization
• Multi-regime Optimization
• Gaussian Processes
• Uncertainty Calibration
• Dirichlet Process

Innovation

• Bayesian Optimization
• Dirichlet Process Mixture
• Gaussian Process
• Multi-regime Optimization
• Collapsed Gibbs Sampling