e-SimFT: Alignment of Generative Models with Simulation Feedback for Pareto-Front Design Exploration

2025-02-04

AI Summary

Generative models struggle to efficiently explore high-quality Pareto fronts in multi-objective engineering design. Method: This paper proposes a simulation-driven generative model alignment framework—the first to adapt the large language model (LLM) preference alignment paradigm to engineering generative modeling. It introduces an ε-constraint-based fine-grained sampling strategy and leverages simulation feedback for automated, scalable Pareto front construction. The method integrates deep generative modeling, multi-objective optimization, and a reinforcement learning from human feedback (RLHF) variant, using simulation signals—rather than human annotations—for model fine-tuning. Contribution/Results: Evaluated on multiple engineering design benchmarks, the approach significantly improves the quality, coverage, and uniformity of the Pareto solution set, outperforming existing generative multi-objective alignment methods.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Deep generative models have recently shown success in solving complex engineering design problems where models predict solutions that address the design requirements specified as input. However, there remains a challenge in aligning such models for effective design exploration. For many design problems, finding a solution that meets all the requirements is infeasible. In such a case, engineers prefer to obtain a set of Pareto optimal solutions with respect to those requirements, but uniform sampling of generative models may not yield a useful Pareto front. To address this gap, we introduce a new framework for Pareto-front design exploration with simulation fine-tuned generative models. First, the framework adopts preference alignment methods developed for Large Language Models (LLMs) and showcases the first application in fine-tuning a generative model for engineering design. The important distinction here is that we use a simulator instead of humans to provide accurate and scalable feedback. Next, we propose epsilon-sampling, inspired by the epsilon-constraint method used for Pareto-front generation with classical optimization algorithms, to construct a high-quality Pareto front with the fine-tuned models. Our framework, named e-SimFT, is shown to produce better-quality Pareto fronts than existing multi-objective alignment methods.

Problem

• Aligns generative models for Pareto-front design
• Uses simulation for scalable and accurate feedback
• Proposes epsilon-sampling for high-quality Pareto fronts

Innovation

• Simulation fine-tuned generative models
• Preference alignment methods adaptation
• Epsilon-sampling for Pareto front