Dynamic Context Scheduling: Learning Beyond the Static Universe

2026-08-21

AI Summary

研究通过动态上下文调度方法改进强化学习中的环境适应性，提出DYNAMICCARLENV框架，并在多个环境中验证了其优于静态上下文基线的效果。

Abstract

We study dynamic context scheduling as a training instrument for contextual re- inforcement learning. Rather than treating intra-episode context variation as a deployment reality, we treat it as a controlled shaping mechanism. Thereby, context evolves within each training episode according to a predetermined schedule, expos- ing the policy to a richer and more temporally structured region of the environment parameter space. We introduce DYNAMICCARLENV, a framework that wraps contextual environments with pluggable schedule families, such as sinusoidal off- sets or cosine annealing. Across CartPole, BipedalWalker and VehicleRacing with CARL contextualization, we show that dynamic schedules match or outperform static context baselines in the out-of-distribution (OOD) regimes. Interestingly, for the more complex BipedalWalker and VehicleRacing environments we also achieve higher in-distribution (ID) evaluation performance. Preliminary findings indicate that automatic search for multi-stage curricula can successfully discover schedules that improve generalization, performing comparably to extensive grid search over single-stage schedulers.

Problem

• Dynamic Context Scheduling
• Reinforcement Learning
• Out-of-Distribution Generalization

Innovation

• dynamic context scheduling
• DYNAMICCARLENV
• temporally structured environment
• out-of-distribution performance
• automatic curriculum search