AutoOR: Scalably Post-training LLMs to Autoformalize Operations Research Problems

2026-04-17

AI Summary

This work proposes a post-training framework that integrates synthetic data generation with reinforcement learning to enable large language models to automatically translate natural language descriptions of operations research (OR) problems into formal optimization models—including linear, mixed-integer, and nonlinear formulations—thereby substantially reducing reliance on specialized OR expertise. The approach innovatively leverages solver feedback as a reward signal and introduces a curriculum-based reinforcement learning strategy tailored for nonlinear dynamical problems, achieving, for the first time, effective automated modeling in this challenging domain. Evaluated on an 8B-parameter model, the method matches or exceeds state-of-the-art performance across six standard OR benchmarks, rivaling results from significantly larger models, and boosts solution accuracy on nonlinear dynamics tasks from near 0% to within solvable ranges.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Reinforcement Learning

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Optimization problems are central to decision-making in manufacturing, logistics, scheduling, and other industrial settings. Translating complicated descriptions of these problems into solver-ready formulations requires specialized operations research (OR) expertise, making it hard to scale. We present AutoOR, a scalable synthetic data generation and reinforcement learning pipeline that trains LLMs to autoformalize optimization problems specified in natural language across linear, mixed-integer, and non-linear categories. AutoOR generates verified training data from standard optimization forms and uses solver execution feedback as the reward signal for RL post-training. AutoOR applied to an 8B model achieves state-of-the-art or competitive results across six established OR benchmarks, matching significantly larger frontier models. For a non-linear problem class involving physical dynamics, where frontier models score near 0%, we introduce a curriculum RL strategy that bootstraps from limited initial training data to make this class tractable for post-training. We believe that methods such as AutoOR can significantly accelerate industrial decision-making with AI.

Problem

• autoformalization
• optimization problems
• operations research
• natural language to formal specification
• scalable AI

Innovation

• autoformalization
• reinforcement learning
• synthetic data generation
• operations research
• curriculum learning