OptMATH: A Scalable Bidirectional Data Synthesis Framework for Optimization Modeling

2025-02-16

AI Summary

Low-quality and small-scale optimization modeling datasets severely hinder large language models’ robustness and generalization in translating between natural language (NL) and mathematical formulations (MF). To address this, we propose the first high-fidelity, bidirectional NL↔MF synthetic paradigm tailored for optimization modeling: (1) using MFs as seeds to controllably generate NL problems of varying complexity; (2) ensuring fidelity via bidirectional NL↔MF translation, forward modeling validation, and rejection sampling; and (3) systematically identifying long-range, challenging instances to construct the first benchmark supporting ultra-long, high-complexity optimization modeling. Our approach achieves state-of-the-art performance across multiple modeling benchmarks—outperforming NL4OPT, MAMO, and others—on models ranging from 0.5B to 32B parameters. We publicly release both the synthetic dataset and the new evaluation benchmark.

Technology Category

• Natural Language Processing: Learning & Optimization for NLP
• Machine Learning: Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Despite the rapid development of large language models (LLMs), a fundamental challenge persists: the lack of high-quality optimization modeling datasets hampers LLMs' robust modeling of practical optimization problems from natural language descriptions (NL). This data scarcity also contributes to the generalization difficulties experienced by learning-based methods. To address these challenges, we propose a scalable framework for synthesizing a high-quality dataset, named OptMATH. Starting from curated seed data with mathematical formulations (MF), this framework automatically generates problem data (PD) with controllable complexity. Then, a back-translation step is employed to obtain NL. To verify the correspondence between the NL and the PD, a forward modeling step followed by rejection sampling is used. The accepted pairs constitute the training part of OptMATH. Then a collection of rejected pairs is identified and further filtered. This collection serves as a new benchmark for optimization modeling, containing difficult instances whose lengths are much longer than these of NL4OPT and MAMO. Through extensive experiments, we demonstrate that models of various sizes (0.5B-32B parameters) trained on OptMATH achieve superior results on multiple modeling benchmarks, thereby validating the effectiveness and scalability of our approach.

Problem

• Lack of high-quality optimization datasets
• Generalization difficulties in learning-based methods
• Synthesis of complex problem data from natural language

Innovation

• Bidirectional data synthesis
• Controllable complexity generation
• Rejection sampling validation