Design Editing for Offline Model-based Optimization

2024-05-22

arXiv.org

AI Summary

Offline model-based optimization (MBO) suffers from out-of-distribution (OOD) prediction: surrogate models often assign spuriously high scores to invalid designs. This work proposes a diffusion-prior-guided design editing framework—the first to integrate diffusion models into offline MBO. Leveraging controllable noise injection and denoising, it calibrates gradient-optimized pseudo-designs back into the underlying data distribution, thereby mitigating OOD bias. The method jointly incorporates surrogate-gradient guidance, diffusion-based prior modeling, and noise-aware editing—operating entirely without environment interaction—to enhance both design validity and alignment between predicted and true objective scores. Evaluated on seven standard offline MBO benchmarks, it achieves state-of-the-art performance, significantly outperforming existing approaches across all tasks.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision
• Machine Learning: Optimization

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

Offline model-based optimization (MBO) aims to maximize a black-box objective function using only an offline dataset of designs and scores. These tasks span various domains, such as robotics, material design, and protein and molecular engineering. A common approach involves training a surrogate model using existing designs and their corresponding scores, and then generating new designs through gradient-based updates with respect to the surrogate model. This method suffers from the out-of-distribution issue, where the surrogate model may erroneously predict high scores for unseen designs. To address this challenge, we introduce a novel method, Design Editing for Offline Model-based Optimization (DEMO), which leverages a diffusion prior to calibrate overly optimized designs. DEMO first generates pseudo design candidates by performing gradient ascent with respect to a surrogate model. While these pseudo design candidates contain information beyond the offline dataset, they might be invalid or have erroneously high predicted scores. Therefore, to address this challenge while utilizing the information provided by pseudo design candidates, we propose an editing process to refine these pseudo design candidates. We introduce noise to the pseudo design candidates and subsequently denoise them with a diffusion prior trained on the offline dataset, ensuring they align with the distribution of valid designs. Empirical evaluations on seven offline MBO tasks show that, with properly tuned hyperparameters, DEMOs score is competitive with the best previously reported scores in the literature.

Problem

• Maximize black-box function using offline dataset
• Address out-of-distribution issue in surrogate models
• Refine pseudo designs with diffusion prior calibration

Innovation

• Uses gradient ascent with surrogate model
• Leverages diffusion prior for calibration
• Edits designs via noise addition and denoising