Groupoid-Based Internal State Representations for Reinforcement Learning with Local Symmetries

2026-09-11

AI Summary

本文提出了一种基于群胚的强化学习框架，通过动态发现局部对称性来提高样本效率和收敛速度，解决了传统方法在处理局部、上下文依赖规律时的局限。

Abstract

Symmetries play a central role in reducing the complexity of reinforcement learning problems, yet most existing approaches rely on fixed group actions or predefined state abstractions. Classical reinforcement learning algorithms typically assume a globally structured Markov decision process with uniformly applicable actions and transitions, an assumption that limits their ability to exploit modularity and local, context-dependent regularities present in many realistic environments. We propose a reinforcement learning framework using groupoids to capture local, state-dependent symmetries and support the dy- namic discovery of equivalence structures during interaction. The agent maintains orbit representatives together with transporters that map raw states to canonical forms, enabling learning and decision-making to be performed in a symmetry-reduced space while preserving local distinctions. Empirical results demonstrate that the proposed groupoid-based approach improves sample efficiency and convergence in dense and large-scale environments exhibiting strong partial symmetries, yielding substantial performance gains over standard Q-learning. These findings show that dynamically exploiting local symmetry provides a practical and mathematically principled route to scalable and generalisable reinforcement learning.

Problem

• local symmetries
• reinforcement learning
• Markov decision process
• state abstractions

Innovation

• groupoid-based approach
• local symmetries
• dynamic discovery of equivalence structures
• symmetry-reduced space
• sample efficiency