Distilling a Modular Reservoir Through a Genomic Bottleneck

2026-06-20

AI Summary

This work proposes a biologically inspired approach to generate sparse, modular recurrent neural networks from highly compressed genotypic encodings for efficiently solving complex temporal tasks. By integrating genotypic compression with modular reservoir computing for the first time, the method employs a hypernetwork to learn a structured generative process and introduces a dual developmental-evolutionary learning paradigm augmented with curriculum-based meta-learning. The resulting networks exhibit strong modularity and sparsity, achieving high performance and robustness with minimal task-specific training. Empirical evaluations demonstrate that the generated architectures significantly outperform existing methods across multiple challenging temporal benchmarks, highlighting the efficacy of combining developmental principles with evolutionary and meta-learning strategies in neural architecture design.

Abstract

The intricate structures of biological neural networks largely emerge during development, guided by a comparatively compressed blueprint encoded in the genome. The connectivity that emerges from this decoding process is rich in structure, and already equips the organism with functional modules upon birth. This initial structure serves as a scaffold that can be gradually refined and fine-tuned through lifelong experience, via a variety of plasticity mechanisms. Drawing inspiration from this interaction between evolutionary and developmental modes of learning, we use hypernetworks to learn a compressed generative process that generates the connectivity of a modular reservoir. We show that this marriage between curriculum-based meta-learning and modular reservoir computing can generate sparse recurrent networks that solve difficult temporal tasks with minimal training and without concessions to robustness.

Problem

• modular reservoir
• genomic bottleneck
• developmental learning
• sparse recurrent networks
• temporal tasks

Innovation

• modular reservoir computing
• hypernetworks
• curriculum-based meta-learning
• genomic bottleneck
• sparse recurrent networks