PoE-World: Compositional World Modeling with Products of Programmatic Experts

2025-05-16

AI Summary

To address the challenges of few-shot learning, dynamic model updating, and cross-task generalization in complex non-grid environments (e.g., Atari games), this paper proposes PoE-World: the first world modeling framework formalized as a Large Language Model (LLM)-driven Product of Exponentially Weighted Program Experts, representing environment dynamics via composable, interpretable probabilistic programs. Our method integrates LLM-based program synthesis, probabilistic program modeling, and model-predictive control to construct source-code-based world models. Evaluated on Pong and Montezuma’s Revenge, PoE-World achieves accurate dynamics modeling from only a few observations—substantially outperforming existing deep world models. The learned models support incremental knowledge updating, embedded planning, and zero-shot generalization across levels. This work establishes a novel paradigm for AI agents that are few-shot adaptable, inherently interpretable, and evolutionarily maintainable.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Multiagent Systems: Adversarial Agents

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Learning how the world works is central to building AI agents that can adapt to complex environments. Traditional world models based on deep learning demand vast amounts of training data, and do not flexibly update their knowledge from sparse observations. Recent advances in program synthesis using Large Language Models (LLMs) give an alternate approach which learns world models represented as source code, supporting strong generalization from little data. To date, application of program-structured world models remains limited to natural language and grid-world domains. We introduce a novel program synthesis method for effectively modeling complex, non-gridworld domains by representing a world model as an exponentially-weighted product of programmatic experts (PoE-World) synthesized by LLMs. We show that this approach can learn complex, stochastic world models from just a few observations. We evaluate the learned world models by embedding them in a model-based planning agent, demonstrating efficient performance and generalization to unseen levels on Atari's Pong and Montezuma's Revenge. We release our code and display the learned world models and videos of the agent's gameplay at https://topwasu.github.io/poe-world.

Problem

• Learning world models for AI adaptation in complex environments
• Overcoming data-hungry deep learning with programmatic expert synthesis
• Enabling generalization in non-gridworld domains from sparse observations

Innovation

• Uses LLMs to synthesize programmatic experts
• Represents world models as weighted product
• Learns stochastic models from few observations