Learning Local Constraints for Reinforcement-Learned Content Generators

2026-05-13

AI Summary

Existing procedural content generation methods struggle to simultaneously preserve local visual style and satisfy global properties such as playability: constraint-based approaches lack global guarantees, while reinforcement learning (RL) methods often suffer from insufficient visual consistency. This work proposes a novel framework that integrates Wave Function Collapse (WFC) with Procedural Content Generation via Reinforcement Learning (PCGRL). By embedding the local constraints learned by WFC into the RL action space and designing compatible input representations and an initial state collapse strategy, the method enables joint optimization of local aesthetics and global objectives. To our knowledge, this is the first approach to effectively combine learned local constraints with RL, successfully generating levels for Lode Runner–style puzzle-platform games that are both visually coherent and highly playable, thereby demonstrating its efficacy and practicality.

Abstract

Constraint-based game content generators that learn local constraints from existing content, such as Wave Function Collapse (WFC), can generate visually satisfying game levels but face challenges in guaranteeing global properties, such as playability. On the other hand, reinforcement-learning trained generators can guarantee global properties -- because such properties can easily be included in reward functions -- but the results can be visually dissatisfying. In this paper, we explore ways to combine these methods. Specifically, we constrain the action space of a PCGRL generator with constraints learned by WFC, effectively allowing the PCGRL generator to achieve global properties while forced to adhere to local constraints. To better analyze how this hybrid content generation method operates, we vary the number and type of inputs, and we test whether to randomly collapse the starting state and exclude rare patterns. While the method is sensitive to hyperparameter tuning, the best of our trained generators produce visually satisfying and playable puzzle-platform game levels -- such as Lode Runner levels -- with desired global properties.

Problem

• Procedural Content Generation
• Local Constraints
• Global Properties
• Reinforcement Learning
• Wave Function Collapse

Innovation

• Wave Function Collapse
• Procedural Content Generation via Reinforcement Learning
• Local Constraints
• Global Properties
• Hybrid Content Generation