A Model-Driven Approach for Developing Families of Reinforcement Learning Environments

2026-06-18

AI Summary

This work addresses the limitations of traditional reinforcement learning (RL) environment families, which are typically handcrafted and thus labor-intensive, error-prone, and difficult to scale. To overcome these challenges, the authors propose a model-driven approach that integrates model-driven engineering with RL environment generation for the first time. Their method employs a hybrid genetic algorithm combining global population-based search with heuristic local search, and expresses variation and constraints through model transformations. By leveraging a model transformation engine, the approach automatically constructs diverse, constraint-compliant environment families, effectively supporting advanced training paradigms such as curriculum learning. Experimental evaluation in a wildfire mitigation scenario demonstrates the method’s efficacy, yielding significantly improved training efficiency for RL agents.

Abstract

Virtual training environments are software-intensive systems in which reinforcement learning (RL) agents learn, adapt, and demonstrate meaningful behavior. Virtual training environments offer a safe and cost-efficient alternative to training agents in real-world settings. However, to converge, most realistic RL problems require training in multiple, mostly similar but slightly different environments - i.e., families of environment variants. The typical development process of environment families is a labor-intensive and error-prone manual endeavor that does not scale well. To alleviate these issues, in this paper, we propose a model-driven approach for developing families of RL training environments. To obtain the family of environments, we develop an approach and prototype tool. In our approach, a hybrid genetic algorithm - a combination of population-based global search and heuristic local search - generates environment families. Mutations and constraints are expressed as model transformations and are operationalized into a search process by a state-of-the-art model transformation engine. We demonstrate the soundness of our approach in a wildfire mitigation scenario and curriculum learning - a particular learning paradigm that relies on environment families.

Problem

• reinforcement learning
• training environments
• environment families
• model-driven development
• virtual training

Innovation

• model-driven development
• reinforcement learning environments
• genetic algorithm
• model transformation
• curriculum learning