Learning Game-Playing Agents with Generative Code Optimization

2025-08-26

AI Summary

This work addresses the challenge of enabling game agents to perform long-horizon, complex reasoning and achieve low-human-intervention autonomous optimization. We propose a generative code optimization framework that explicitly represents policies as executable, evolvable Python programs. Leveraging large language models (LLMs), the agent iteratively refines its policy by jointly analyzing program execution traces and natural-language feedback. Our core contribution is replacing conventional neural network parameterizations with programmable policy representations, endowing agents with explicit logical reasoning, debugging, and evolutionary capabilities. Evaluated on the Atari benchmark, our approach achieves performance competitive with state-of-the-art deep reinforcement learning methods, while reducing training time by ~40% and environment interactions by ~60%. Crucially, it significantly diminishes reliance on handcrafted reward functions and extensive trial-and-error interaction. These results demonstrate the promise of programmatic policy representations for sample-efficient, interpretable, and human-aligned agent learning.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Search and Optimization: Learning to Search
• Multiagent Systems: Adversarial Agents

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We present a generative optimization approach for learning game-playing agents, where policies are represented as Python programs and refined using large language models (LLMs). Our method treats decision-making policies as self-evolving code, with current observation as input and an in-game action as output, enabling agents to self-improve through execution traces and natural language feedback with minimal human intervention. Applied to Atari games, our game-playing Python program achieves performance competitive with deep reinforcement learning (RL) baselines while using significantly less training time and much fewer environment interactions. This work highlights the promise of programmatic policy representations for building efficient, adaptable agents capable of complex, long-horizon reasoning.

Problem

• Learning game-playing agents via generative code optimization
• Refining Python program policies using large language models
• Achieving competitive performance with fewer environment interactions

Innovation

• Generative code optimization for agent learning
• Self-evolving Python policies using LLMs
• Programmatic agents with minimal environment interaction