Learning Design-Score Manifold to Guide Diffusion Models for Offline Optimization

2025-06-06

AI Summary

Offline black-box system optimization faces challenges of limited pre-collected data, high evaluation costs, and poor out-of-distribution (OOD) generalization. Method: We propose ManGO—a manifold-guided optimization framework that jointly models the design-performance score manifold to eliminate online interaction; integrates derivative-free guidance with adaptive denoising path scaling to enhance OOD generalization; and unifies conditional diffusion generation, manifold learning, and inference-time dynamic path optimization. Contribution/Results: Evaluated across five domains—synthetic benchmarks, robot control, materials design, DNA sequence generation, and engineering optimization—ManGO consistently outperforms 34 state-of-the-art baselines, achieving high-quality design generation without any function queries.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Optimization
• Constraint Satisfaction and Optimization: Constraint Optimization

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Optimizing complex systems, from discovering therapeutic drugs to designing high-performance materials, remains a fundamental challenge across science and engineering, as the underlying rules are often unknown and costly to evaluate. Offline optimization aims to optimize designs for target scores using pre-collected datasets without system interaction. However, conventional approaches may fail beyond training data, predicting inaccurate scores and generating inferior designs. This paper introduces ManGO, a diffusion-based framework that learns the design-score manifold, capturing the design-score interdependencies holistically. Unlike existing methods that treat design and score spaces in isolation, ManGO unifies forward prediction and backward generation, attaining generalization beyond training data. Key to this is its derivative-free guidance for conditional generation, coupled with adaptive inference-time scaling that dynamically optimizes denoising paths. Extensive evaluations demonstrate that ManGO outperforms 24 single- and 10 multi-objective optimization methods across diverse domains, including synthetic tasks, robot control, material design, DNA sequence, and real-world engineering optimization.

Problem

• Optimizing complex systems without known rules or costly evaluations
• Overcoming limitations of conventional offline optimization methods
• Learning design-score manifold for improved generalization and performance

Innovation

• Learns design-score manifold holistically
• Unifies forward prediction and backward generation
• Uses derivative-free guidance with adaptive scaling