When Does Neuroevolution Outcompete Reinforcement Learning in Transfer Learning Tasks?

2025-05-28

AI Summary

This work investigates the applicability boundaries of neuroevolution (NE) versus reinforcement learning (RL) in transfer learning, specifically examining NE’s capacity to mitigate RL’s brittleness and catastrophic forgetting. Method: We introduce two structured transfer benchmarks—“Stepwise Logic Circuits” and “EcoRobot,” a multi-morphology robotic platform—designed to emphasize modular redundancy and morphological variability, thereby addressing the lack of quantitative evaluation frameworks for NE transfer capability. Our approach integrates evolutionary architecture optimization, Brax-based physics simulation extensions, curriculum-driven progressive task design, and multi-morphology control modeling. Contribution/Results: Experiments demonstrate that, under conditions of structural task similarity and morphological reusability, multiple NE methods significantly improve transfer success rates and robustness against catastrophic forgetting—providing the first systematic characterization of the conditions under which NE exhibits superior performance in structured transfer settings.

Technology Category

• Machine Learning: Evolutionary Learning
• Search and Optimization: Evolutionary Computation
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The ability to continuously and efficiently transfer skills across tasks is a hallmark of biological intelligence and a long-standing goal in artificial systems. Reinforcement learning (RL), a dominant paradigm for learning in high-dimensional control tasks, is known to suffer from brittleness to task variations and catastrophic forgetting. Neuroevolution (NE) has recently gained attention for its robustness, scalability, and capacity to escape local optima. In this paper, we investigate an understudied dimension of NE: its transfer learning capabilities. To this end, we introduce two benchmarks: a) in stepping gates, neural networks are tasked with emulating logic circuits, with designs that emphasize modular repetition and variation b) ecorobot extends the Brax physics engine with objects such as walls and obstacles and the ability to easily switch between different robotic morphologies. Crucial in both benchmarks is the presence of a curriculum that enables evaluating skill transfer across tasks of increasing complexity. Our empirical analysis shows that NE methods vary in their transfer abilities and frequently outperform RL baselines. Our findings support the potential of NE as a foundation for building more adaptable agents and highlight future challenges for scaling NE to complex, real-world problems.

Problem

• Compare neuroevolution and reinforcement learning in transfer learning
• Evaluate skill transfer across tasks with increasing complexity
• Assess neuroevolution's robustness and scalability in adaptable agents

Innovation

• Neuroevolution outperforms reinforcement learning in transfer
• Introduces stepping gates and ecorobot benchmarks for evaluation
• Curriculum enables skill transfer across increasing complexity tasks