MiniOpt: Reasoning to Model and Solve General Optimization Problems with Limited Resources

2026-06-24

AI Summary

This work addresses the challenge that large language models struggle to generalize across diverse optimization problems under limited training resources. To overcome this, the authors propose MiniOpt, a novel framework that adopts a “reasoning-to-modeling-to-solving” paradigm, decomposing optimization tasks into structured modeling and executable solver generation, trained end-to-end via reinforcement learning. The key innovation lies in OptReward, a hierarchical reward function that requires no expert demonstrations, combined with an optimization-oriented policy optimization method, significantly enhancing generalization and exploration efficiency for small-scale models. Experimental results demonstrate that MiniOpt, with only 3B parameters, achieves state-of-the-art solution accuracy among models under 10B parameters across a variety of optimization problems and remains competitive even against larger models.

Abstract

Achieving strong optimization generalization across diverse optimization problems while requiring limited training resources remains a challenging problem for optimization-oriented large language models (LLMs). Existing approaches typically rely on large-scale supervised datasets, costly reasoning annotations, and expensive intermediate step verification, resulting in substantial training overhead. To address these challenges, we propose MiniOpt, a reinforcement learning framework that learns to solve optimization problems through an "reasoning-to-model-and-solve" paradigm. MiniOpt decomposes optimization reasoning into structured optimization modeling and executable solver generation. Building upon this paradigm, we introduce OptReward, a reward function with hierarchical score structure that jointly evaluates formulation and solution, enabling effective policy learning without expert demonstrations. We further develop an optimization-oriented policy optimization strategy that improves exploration efficiency and stabilizes reinforcement learning for compact models. Extensive experiments show that MiniOpt-3B exhibits strong optimization generalization across various optimization types, problem scenarios, and task domains. For models with fewer than 10B parameters, MiniOpt series achieves the highest average solving accuracy (SA). For models with more than 10B parameters, MiniOpt still shows competitive performance. These results suggest that optimization-oriented reward design and reinforcement learning provide an effective pathway for developing compact optimization-specialized language models with strong optimization generalization capabilities. The code is available at https://github.com/Hsiang-1/MiniOpt.

Problem

• optimization generalization
• limited training resources
• optimization-oriented LLMs
• reasoning-to-model

Innovation

• reasoning-to-model-and-solve
• OptReward
• reinforcement learning
• optimization generalization
• compact LLMs