Knowledge Reutilization in Meta-Reinforcement Learning

2026-06-16

AI Summary

This work addresses the limitations of existing meta-reinforcement learning approaches, which often couple task inference with policy execution, leading to ambiguous task semantics, low sample efficiency, and poor knowledge transfer across heterogeneous agents. To overcome these issues, the authors propose a decoupled meta-knowledge reuse framework that learns task-level knowledge on simplified dynamical surrogates. Task structures are organized via a Bayesian nonparametric prior, and knowledge is transferred to diverse agents through a semantic-magnitude interface combined with a lightweight temporal adapter. This design enables frozen-task knowledge reuse, substantially improving cross-embodiment transfer efficiency. Experiments demonstrate that the method achieves performance comparable to state-of-the-art baselines using only approximately 23.8% of the interaction data across multiple locomotion agents, while reducing final tracking errors by 94.75%–99.79%.

Abstract

Meta-reinforcement learning enables fast adaptation by extracting shared structure from related tasks, but existing end-to-end methods often couple task inference with embodiment-specific control. This coupling can obscure non-parametric task semantics, reduce sample efficiency, and limit cross-agent reuse. We propose a meta-knowledge reutilization framework that learns task-level knowledge on a dynamics-simplified agent and transfers it to heterogeneous agents. The framework uses a Bayesian non-parametric prior to organize latent task modes and a high-level policy to generate task-level magnitude guidance. To bridge reusable task knowledge with different embodiments, we introduce a semantic-magnitude interface and a lightweight temporal adaptor, which convert frozen meta-knowledge into temporally aligned subgoals for embodiment-specific low-level controllers. Experiments on multiple locomotion agents show that our framework reduces final-step tracking error by 94.75% -- 99.79% compared with recent state-of-the-art baselines and achieves comparable deployment performance with about 23.8% of their interaction data.

Problem

• meta-reinforcement learning
• knowledge reutilization
• cross-agent transfer
• task inference
• embodiment-specific control

Innovation

• meta-reinforcement learning
• knowledge reutilization
• Bayesian non-parametric prior
• cross-agent transfer
• semantic-magnitude interface