🤖 AI Summary
This work addresses the challenge that local large language models struggle to ensure correctness in coding tasks requiring feedback, persistent state, and limited repair opportunities, due to the absence of verification-driven memory and skill reuse mechanisms. The authors propose a lightweight external controller that operates with frozen model weights and integrates abstract syntax tree (AST)-based skill extraction, a fail-fast verification pipeline, verification-guided memory logging, and TD(λ) eligibility traces for delayed credit assignment. Crucially, verification outcomes are transformed into bounded shaping rewards, enabling structured skill reuse and adaptive retrieval. Evaluated on rigorously verified reinforcement learning coding tasks, the approach succeeds in 8 out of 9 experiments, substantially outperforming both self-refinement baselines and GRACE extensions, which achieve zero successes across all trials.
📝 Abstract
Local LLM-based coding agents increasingly work in settings where correctness is earned through execution feedback, persistent state, and bounded repair, not through a single fluent answer. Static retrieval, long-context prompting, self-refinement, execution-feedback repair, and reinforcement learning over model weights each address part of this setting, but they do not jointly provide validation-grounded episodic memory, adaptive retrieval-action selection, delayed credit assignment, and structural skill reuse around a frozen local model. We introduce PYTHALAB-MERA, a lightweight external controller for local validation-conditioned code generation. The frozen language model proposes complete source files; the controller decides which memory records and AST-derived skills should enter the next prompt, validates each candidate through a fail-fast pipeline, converts validation outcomes into bounded shaped rewards, and propagates delayed credit through TD(lambda)-style eligibility traces. We evaluate the implementation as a local CLI artifact on reinforcement-learning coding tasks with strict validation gates. In the measured hard RL setting with three tasks, three repetitions, and a three-attempt budget, PYTHALAB-MERA passed 8/9 strict validations; the self-refinement baseline and the investigated GRACE extension each passed 0/9. These results support a deliberately bounded claim: in this recorded setting, the external memory-and-retrieval controller improved validation success. They do not establish general-purpose code synthesis, state-of-the-art performance, formal program correctness, or formal safety.