Autonomous Behavior and Whole-Brain Dynamics Emerge in Embodied Zebrafish Agents with Model-based Intrinsic Motivation

2025-05-30

AI Summary

Existing reinforcement learning methods exhibit insufficient exploration capability in sparse- or zero-reward environments, hindering the emergence of animal-like robust autonomy; concurrently, systems neuroscience has long neglected the neural mechanisms underlying autonomous behavior, favoring externally reward-driven paradigms. Method: We introduce an embodied zebrafish larva agent featuring a novel intrinsic motivation—3M-Progress—grounded in world-model prediction error and ecological priors, enabling reward-free autonomous exploration. We further develop the first interpretable, goal-directed whole-brain glial computational model, integrating model-predictive control, world-model learning, neuro-behavioral joint modeling, and whole-brain calcium imaging analysis. Results: Our approach successfully reproduces natural behavioral patterns and glial dynamics, achieves significantly higher variance explanation than state-of-the-art intrinsic motivation methods, and establishes, for the first time, a testable bridge between computational autonomy and systems neuroscience.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Autonomy is a hallmark of animal intelligence, enabling adaptive and intelligent behavior in complex environments without relying on external reward or task structure. Existing reinforcement learning approaches to exploration in sparse reward and reward-free environments, including class of methods known as intrinsic motivation, exhibit inconsistent exploration patterns and thus fail to produce robust autonomous behaviors observed in animals. Moreover, systems neuroscience has largely overlooked the neural basis of autonomy, focusing instead on experimental paradigms where animals are motivated by external reward rather than engaging in unconstrained, naturalistic and task-independent behavior. To bridge these gaps, we introduce a novel model-based intrinsic drive explicitly designed to capture robust autonomous exploration observed in animals. Our method (3M-Progress) motivates naturalistic behavior by tracking divergence between the agent's current world model and an ethological prior. We demonstrate that artificial embodied agents trained with 3M-Progress capture the explainable variance in behavioral patterns and whole-brain neural-glial dynamics recorded from autonomously-behaving larval zebrafish, introducing the first goal-driven, population-level model of neural-glial computation. Our findings establish a computational framework connecting model-based intrinsic motivation to naturalistic behavior, providing a foundation for building artificial agents with animal-like autonomy.

Problem

• Bridging gap between animal autonomy and artificial agent behavior
• Modeling neural-glial dynamics in autonomous zebrafish behavior
• Developing intrinsic motivation for naturalistic exploration in agents

Innovation

• Model-based intrinsic drive for autonomous exploration
• Tracking divergence between world model and ethological prior
• Goal-driven population-level neural-glial computation model