OptSkills: Learning Generalizable Optimization Skills from Problem Archetypes via Cluster-Based Distillation

2026-05-28

AI Summary

This work addresses the limited generalization of current large language models (LLMs) in optimization tasks, particularly when facing superficially rephrased problems or novel formulations. To overcome this, the authors propose a prototype-centered skill learning and reasoning framework that first clusters problems to uncover their underlying structural prototypes. Within each cluster, diverse modeling and solving strategies are explored and then distilled into reusable, workflow-level optimization skills. By integrating LLMs with clustering, trajectory distillation, and skill modeling, the system achieves effective generalization both within and outside the training distribution while enabling continual skill expansion. Experiments demonstrate strong performance across multiple optimization benchmarks, yielding a micro-averaged accuracy of 68.27%, outperforming DeepSeek-V3.2-Thinking by 4.53% on MIPLIB-NL (achieving 26.91%), and attaining 72.79% on the NLCO out-of-distribution benchmark.

Abstract

Leveraging Large Language Models (LLMs) to automatically formulate and solve optimization problems from natural language has emerged as an efficient paradigm for automated optimization. However, existing methods still exhibit limited generalization: they are sensitive to superficial narrative variations, reuse experience mainly at the case level, and struggle to adapt to shifted or emerging problem types. We propose OptSkills, an archetype-centric skill learning and reasoning agent system for optimization modeling and solving. To improve robust generalization, our system clusters problems by their underlying archetypes rather than surface narratives. To improve in-distribution generalization, it explores diverse modeling paradigms and solver configurations within each cluster, then distills successful trajectories into reusable workflow-level skills. To improve out-of-distribution generalization, it refines existing skills or expands the skill library using newly obtained trajectories. Our system achieves a state-of-the-art micro-averaged accuracy of 68.27% on datasets encompassing diverse problem types and scenarios. In addition, on MIPLIB-NL, a highly challenging large-scale and high-dimensional benchmark, it achieves 26.91% accuracy, outperforming DeepSeek-V3.2-Thinking by 4.53%. After skill learning on Nano-CO, it reaches 72.79% on the OOD NLCO benchmark. Code and skills are available at https://github.com/fujiwaranoM0kou/OptSkills.

Problem

• generalization
• optimization
• large language models
• problem archetypes
• out-of-distribution

Innovation

• archetype-based clustering
• skill distillation
• optimization modeling
• generalizable reasoning
• workflow-level skills