A Neuroscience-Inspired Dual-Process Model of Compositional Generalization

2025-07-24

AI Summary

Systematic compositional generalization remains a fundamental bottleneck in AI, whereas humans achieve it through coordinated hippocampal (HPC) and prefrontal cortical (PFC) mechanisms. Inspired by neuroscience’s dual-process theory, we propose MIRAGE—the first AI framework that explicitly models the HPC–PFC functional circuit as a dual-path architecture: a meta-trained Transformer serves as a neural decomposer, coupled with a Schema Engine that iteratively decomposes patterns and explicitly extracts, stores, and applies symbolic rules—entirely without fine-tuning or parameter updates. On all task splits of the SCAN benchmark, MIRAGE achieves >99% zero-shot accuracy using only a frozen 1.19M-parameter pure-Transformer model. This marks the first demonstration of zero-shot systematic generalization in a small-scale, purely Transformer-based architecture, bridging neurocognitive principles with scalable, interpretable AI design.

Technology Category

• Machine Learning: Neuro-Symbolic Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Systematic compositional generalization - constructing and understanding novel combinations of known building blocks - remains a core challenge for AI systems. Human cognition achieves this flexibility via the interplay of the hippocampus (HPC) and prefrontal cortex (PFC): the hippocampus rapidly encodes episodes, and the prefrontal cortex consolidates them into reusable schemas for reasoning. Drawing on these insights, we present MIRAGE (Meta-Inference with Rules and Abstractions from Generalized Experience), a framework that achieves systematic generalization on compositional tasks. MIRAGE has two interacting modules mirroring the brain's deliberative HPC-PFC loop and intuitive neocortical pattern recognition. (1) The meta-trained Transformer Neural Decomposer, paralleling neocortical "System 1" computation, is trained on a task-agnostic stream of randomly sampled compositional grammars and applies one decomposition step per pass, with successive passes iteratively refining the sequence representation. (2) The Schema Engine, analogous to the HPC-PFC "System 2" loop, dynamically extracts, ranks, and applies reusable schemas, storing variable bindings in episodic memory and expanding them when needed. By explicitly equipping the Transformer component of MIRAGE with actively managed schematic structures, our model performs systematic compositional operations through explicit schema application and transformation, relying solely on frozen weights when solving entirely novel tasks. This approach demonstrates systematic compositional generalization on the SCAN benchmark, achieving > 99% accuracy on all task splits with only 1.19M parameters in the transformer module. Ablation studies confirm that MIRAGE's systematicity critically depends on the quality of extracted schemas and the model's iterative refinement process.

Problem

• Addresses systematic compositional generalization in AI systems
• Models human hippocampal-prefrontal cortex interplay for flexibility
• Achieves high accuracy on SCAN benchmark with minimal parameters

Innovation

• Dual-module system mimicking brain's HPC-PFC interaction
• Meta-trained Transformer Neural Decomposer for iterative refinement
• Schema Engine dynamically extracts and applies reusable schemas