Multi-Objective Multi-Fidelity Bayesian Optimization with Causal Priors

2026-01-31

AI Summary

This work addresses the inefficiency in multi-objective multi-fidelity Bayesian optimization caused by poor alignment between low-fidelity surrogates and high-fidelity objectives. To this end, it introduces causal inference into the framework for the first time, constructing a structural causal model to characterize the causal relationships among inputs, fidelity levels, and objectives. Building on this model, the authors develop a multi-fidelity probabilistic surrogate and a novel acquisition function—causal hypervolume knowledge gradient—that leverages interventional effects to enhance the utility of low-fidelity data. This approach significantly improves the approximation of the high-fidelity Pareto front. Extensive experiments on synthetic and real-world benchmarks from robotics, automated machine learning, and healthcare demonstrate that the proposed method consistently outperforms existing state-of-the-art techniques in both sample efficiency and optimization performance.

Technology Category

• Machine Learning: Causal Learning
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

Multi-fidelity Bayesian optimization (MFBO) accelerates the search for the global optimum of black-box functions by integrating inexpensive, low-fidelity approximations. The central task of an MFBO policy is to balance the cost-efficiency of low-fidelity proxies against their reduced accuracy to ensure effective progression toward the high-fidelity optimum. Existing MFBO methods primarily capture associational dependencies between inputs, fidelities, and objectives, rather than causal mechanisms, and can perform poorly when lower-fidelity proxies are poorly aligned with the target fidelity. We propose RESCUE (REducing Sampling cost with Causal Understanding and Estimation), a multi-objective MFBO method that incorporates causal calculus to systematically address this challenge. RESCUE learns a structural causal model capturing causal relationships between inputs, fidelities, and objectives, and uses it to construct a probabilistic multi-fidelity (MF) surrogate that encodes intervention effects. Exploiting the causal structure, we introduce a causal hypervolume knowledge-gradient acquisition strategy to select input-fidelity pairs that balance expected multi-objective improvement and cost. We show that RESCUE improves sample efficiency over state-of-the-art MF optimization methods on synthetic and real-world problems in robotics, machine learning (AutoML), and healthcare.

Problem

• Multi-fidelity Bayesian Optimization
• Causal Priors
• Multi-Objective Optimization
• Sample Efficiency
• Black-box Optimization

Innovation

• Causal Inference
• Multi-Fidelity Bayesian Optimization
• Structural Causal Model
• Multi-Objective Optimization
• Knowledge Gradient