Skill Expansion and Composition in Parameter Space

2025-02-09

AI Summary

Existing continual learning approaches suffer from low training efficiency and poor reusability of previously acquired knowledge during skill expansion. This paper addresses the need for autonomous agents to continually evolve capabilities by proposing a parameter-space framework for skill expansion and composition. Our method introduces: (1) a modular skill library built upon Low-Rank Adaptation (LoRA), enabling plug-and-play skill growth; and (2) a dynamic context-aware activation mechanism that supports direct, parameter-level cross-skill composition and gated fusion. Evaluated on D4RL, DSRL, and the DeepMind Control Suite, the approach significantly improves multi-objective policy synthesis, dynamics transfer, and continual policy evolution. Results demonstrate the effectiveness of parameter-level knowledge reuse and structured skill evolution, advancing continual reinforcement learning for autonomous agents.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Multiagent Systems: Multiagent Learning
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Humans excel at reusing prior knowledge to address new challenges and developing skills while solving problems. This paradigm becomes increasingly popular in the development of autonomous agents, as it develops systems that can self-evolve in response to new challenges like human beings. However, previous methods suffer from limited training efficiency when expanding new skills and fail to fully leverage prior knowledge to facilitate new task learning. In this paper, we propose Parametric Skill Expansion and Composition (PSEC), a new framework designed to iteratively evolve the agents' capabilities and efficiently address new challenges by maintaining a manageable skill library. This library can progressively integrate skill primitives as plug-and-play Low-Rank Adaptation (LoRA) modules in parameter-efficient finetuning, facilitating efficient and flexible skill expansion. This structure also enables the direct skill compositions in parameter space by merging LoRA modules that encode different skills, leveraging shared information across skills to effectively program new skills. Based on this, we propose a context-aware module to dynamically activate different skills to collaboratively handle new tasks. Empowering diverse applications including multi-objective composition, dynamics shift, and continual policy shift, the results on D4RL, DSRL benchmarks, and the DeepMind Control Suite show that PSEC exhibits superior capacity to leverage prior knowledge to efficiently tackle new challenges, as well as expand its skill libraries to evolve the capabilities. Project website: https://ltlhuuu.github.io/PSEC/.

Problem

• Enhancing autonomous agents' skill evolution
• Efficiently integrating prior knowledge for new tasks
• Dynamic skill composition in parameter space

Innovation

• Parametric Skill Expansion and Composition
• Low-Rank Adaptation modules
• Context-aware skill activation