ConcoLixir: Reactive LLM Discovery Oracles for Python Concolic Testing

2026-06-24

AI Summary

This work addresses the limitations of Python concolic testing, which often suffers from symbolic degradation due to library calls, intractable semantic operations, and stalled path exploration. The paper proposes the first integration of a large language model (LLM) as a lightweight, reactive oracle within the concolic execution loop. Without replacing the underlying symbolic solver, the LLM leverages execution feedback and path constraints to dynamically generate initial seeds, suggest concrete inputs upon solver failure, and guide targeted exploration toward uncovered code when coverage plateaus. This approach substantially enhances exploration efficacy, particularly across semantic barriers and library boundaries. Experimental results demonstrate average line coverage improvements of 8.6, 15.1, and 17.0 percentage points on synthetic benchmarks, real-world programs, and library-centric targets, respectively, with a total API cost of only \$1.63.

Abstract

Concolic testing combines concrete execution with symbolic constraint solving, but Python programs expose recurring limits. Library calls can cause symbolic variables to downgrade to concrete values. Regular expressions, checksums, parsers, and other semantic operations can be hard to solve, and exploration can plateau on already covered paths. We present ConcoLixir, a reactive LLM extension for Python concolic execution. The LLM acts as a discovery oracle, not a replacement for the solver or a correctness oracle. It generates initial seeds, proposes concrete inputs after solver failures, and targets uncovered code when coverage stalls. Each candidate is executed concolically, and only observed coverage and collected path constraints guide later exploration. Across synthetic, real-world, and library targets, ConcoLixir improves mean line coverage over the baseline concolic tester without an LLM oracle by 8.6, 15.1, and 17.0 percentage points. The gains are strongest near semantic barriers and library boundaries, and the full evaluation costs \$1.63 in API charges. These results show that bounded LLM discovery can complement symbolic reasoning without replacing it.

Problem

• concolic testing
• symbolic execution
• LLM oracle
• code coverage
• semantic barriers

Innovation

• concolic testing
• large language model
• discovery oracle
• symbolic execution
• test coverage