A Continual Offline Reinforcement Learning Benchmark for Navigation Tasks

2025-06-03

AI Summary

To address core challenges in continual reinforcement learning—including catastrophic forgetting, task adaptation, and memory efficiency—this paper introduces the first continual offline RL benchmark tailored to navigation tasks, covering multi-stage video game scenarios. Methodologically, we establish a standardized task suite, dataset, evaluation protocol, and metrics; within an offline RL framework, we integrate task sequencing, experience replay, representation stability constraints, and incremental evaluation techniques. Our key contributions are: (1) filling a critical gap by providing the first navigation-specific continual offline RL benchmark; (2) enabling reproducible assessment of cross-task generalization and long-term memory retention; and (3) releasing open-source implementations compatible with production-grade pipelines, thereby significantly improving the reliability and scalability of baseline algorithm evaluation. (149 words)

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Adversarial Agents
• Planning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Autonomous agents operating in domains such as robotics or video game simulations must adapt to changing tasks without forgetting about the previous ones. This process called Continual Reinforcement Learning poses non-trivial difficulties, from preventing catastrophic forgetting to ensuring the scalability of the approaches considered. Building on recent advances, we introduce a benchmark providing a suite of video-game navigation scenarios, thus filling a gap in the literature and capturing key challenges : catastrophic forgetting, task adaptation, and memory efficiency. We define a set of various tasks and datasets, evaluation protocols, and metrics to assess the performance of algorithms, including state-of-the-art baselines. Our benchmark is designed not only to foster reproducible research and to accelerate progress in continual reinforcement learning for gaming, but also to provide a reproducible framework for production pipelines -- helping practitioners to identify and to apply effective approaches.

Problem

• Addresses continual offline reinforcement learning in navigation tasks
• Tackles catastrophic forgetting and task adaptation challenges
• Provides benchmark for reproducible research in gaming applications

Innovation

• Benchmark for continual offline reinforcement learning
• Addresses catastrophic forgetting and task adaptation
• Reproducible framework for gaming and production