OptArgus: A Multi-Agent System to Detect Hallucinations in LLM-based Optimization Modeling

2026-05-12

AI Summary

This work addresses the challenge of modeling hallucinations in optimization formulations generated by large language models (LLMs), which, despite yielding numerically correct solutions, often suffer from semantic inconsistencies that evade validation via objective values. To tackle this issue, the paper introduces the first fine-grained hallucination taxonomy tailored to optimization modeling, categorizing errors into four types: objective, variable, constraint, and implementation. Building upon this taxonomy, the authors propose OptArgus, a multi-agent detection framework that integrates a command router, expert auditors, and an evidence fusion mechanism. Evaluated on a benchmark of 7,042 samples, OptArgus substantially outperforms single-agent baselines, significantly reducing false positive rates while enhancing both error localization accuracy and the ability to identify hallucinations in naturally generated modeling content.

Abstract

Large language models (LLMs) are increasingly used to translate natural-language optimization problems into mathematical formulations and solver code, but matching the reference objective value is not a reliable test of correctness: an artifact may agree numerically while still changing the underlying optimization semantics. We formulate this issue as \emph{optimization-modeling hallucination detection}, namely structural consistency auditing over the problem description, symbolic model, and solver implementation. We develop, to our knowledge, the first fine-grained hallucination taxonomy specifically for optimization modeling, spanning objective, variable, constraint, and implementation failures. We use this taxonomy to design OptArgus, a multi-agent detector with conductor routing, specialist auditors, and evidence consolidation. To evaluate this setting, we introduce a three-part benchmark suite with $484$ clean artifacts, $1266$ controlled injected artifacts, and $6292$ natural LLM-generated artifacts. Against a matched single-agent baseline, OptArgus produces fewer false alarms on clean artifacts, more accurate top-ranked localization on controlled single-error cases, and stronger detection on natural model outputs. Together, these contributions turn optimization-modeling hallucination detection into a concrete empirical problem and suggest that modular, taxonomy-grounded auditing is a practical route to more reliable optimization modeling.

Problem

• optimization-modeling hallucination
• structural consistency
• LLM-generated artifacts
• mathematical formulation
• solver implementation

Innovation

• optimization-modeling hallucination
• multi-agent detection
• fine-grained taxonomy
• structural consistency auditing
• LLM-based optimization