Admission Without Answers: Label-Free Certification and Experience Learning for LLM-Based Optimization Modeling

2026-08-16

AI Summary

This study addresses the unreliability of knowledge admission in LLM experiential learning from unlabeled streams due to the absence of validation. We propose AdmitOR, a novel admission mechanism based on explicitly calibrated false discovery objectives that achieves high-reliability knowledge filtering through cross-model-family behavioral evidence, parameter-space resampling, and calibrated threshold decision-making. Experiments demonstrate that AdmitOR attains an admission precision of 0.927, reduces poisoning rates eightfold, and achieves a macro-accuracy of 58.4%, significantly outperforming majority voting and execution-success baselines. These results indicate substantial improvements in both skill library precision and generalization performance. Furthermore, this work reveals the impact of benchmark text distortion on transferability, highlighting critical considerations for deploying experiential learning systems in real-world unlabeled environments where validation signals are inherently unavailable.

Abstract

Experience-learning agents for optimization modeling improve by storing verified skills, but existing learners admit knowledge by checking against known answers, which real ticket streams do not provide. The natural label-free alternatives are unreliable: on a 300-problem label-blind stream, admitting every executable model poisons roughly one admission in four, while single-instance agreement accepts models that match at one value but differ elsewhere. We propose AdmitOR, an admission gate built on calibrated external behavioral evidence. Candidates from three model families, prompting strategies, and solver stacks are run on instances resampled from an extracted parameter domain; agreement across the resulting value-function traces is summarized by a cross-family clique, and a calibrated threshold returns accept, abstain, or escalate. The preregistered false-discovery criterion holds on calibration data but not on the wild stream. We report this negative result in full and trace most failures to benchmark texts that do not faithfully encode their labeled instances. Comparing four admission judges on one collection of logs inside a state-of-the-art skill learner, AdmitOR raises admission precision to 0.927, against 0.871 for majority vote and 0.726 for execution success, yielding 3.1x and 8.0x fewer poisoned admissions. Its library is the smallest and attains the highest macro accuracy across five public benchmarks, 58.4 against 54.8 for majority vote and 53.9 for the ground-truth-labeled library. The 3.5-point gain over majority vote is supported by a paired bootstrap and survives correction for a host-side anomaly. To our knowledge, AdmitOR is the first label-free admission mechanism designed around an explicitly calibrated false-discovery target. The transfer failure identifies a necessary condition for extending it to wild streams.

Problem

• Label-Free Admission
• Optimization Modeling
• Experience Learning
• Knowledge Poisoning
• LLM

Innovation

• Label-Free Admission
• AdmitOR
• Calibrated False Discovery
• Experience Learning
• Cross-Family Clique