Large Language Models as Amortized Pareto-Front Generators for Constrained Bi-Objective Convex Optimization

2026-05-12

AI Summary

This work addresses the high computational cost of traditional methods for constrained bi-objective convex optimization, which require solving each instance repeatedly. The authors propose DIPS, a novel framework that leverages large language models (LLMs) as amortized generators of Pareto fronts, enabling direct generation of ordered feasible solution sets from textual problem descriptions via end-to-end fine-tuning. The approach integrates compact discretization encoding, numerically aware token initialization, and a three-stage curriculum optimization strategy to jointly align solution structure, feasibility, and front quality. Using a 7B-parameter LLM accelerated by vLLM, DIPS achieves normalized hypervolume ratios of 95.29%–98.18% across five problem classes, with single-instance inference as fast as 0.16 seconds—significantly outperforming both general-purpose and reasoning-oriented LLM baselines.

Abstract

Generating feasible Pareto fronts for constrained bi-objective continuous optimization is central to multi-criteria decision-making. Existing methods usually rely on iterative scalarization, evolutionary search, or problem-specific solvers, requiring repeated optimization for each instance. We introduce DIPS, an end-to-end framework that fine-tunes large language models as amortized Pareto-front generators for constrained bi-objective convex optimization. Given a textual problem description, DIPS directly outputs an ordered set of feasible continuous decision vectors approximating the Pareto front. To make continuous optimization compatible with autoregressive language modeling, DIPS combines a compact discretization scheme, Numerically Grounded Token Initialization for new numerical tokens, and Three-Phase Curriculum Optimization, which progressively aligns structural validity, feasibility, and Pareto-front quality. Across five families of constrained bi-objective convex problems, a fine-tuned 7B-parameter model achieves normalized hypervolume ratios of 95.29% to 98.18% relative to reference fronts. With vLLM-accelerated inference, DIPS solves one instance in as little as 0.16 seconds and outperforms general-purpose and reasoning LLM baselines under the evaluated setting. These results suggest that LLMs can serve as effective amortized generators for continuous Pareto-front approximation.

Problem

• Pareto front
• constrained bi-objective optimization
• convex optimization
• multi-criteria decision-making
• continuous optimization

Innovation

• Amortized Optimization
• Pareto-Front Generation
• Large Language Models
• Constrained Bi-Objective Optimization
• Numerically Grounded Token Initialization