A Deep Reinforcement Learning Framework for Closed-loop Guidance of Fish Schools via Virtual Agents

2026-03-30

AI Summary

This study addresses the challenge of simultaneously achieving directional control and maintaining social cohesion when guiding fish schools in closed-loop systems. The authors propose a novel framework that integrates deep reinforcement learning with real biological interactions: a proximal policy optimization (PPO) algorithm trains virtual agents in simulation to optimize a composite reward function that balances guidance objectives with group coherence, enabling online closed-loop guidance in experiments with red-nosed tetras (*Hemigrammus rhodostomus*). This work represents the first application of deep reinforcement learning to the physical steering of live fish groups and systematically evaluates the influence of visual stimulus parameters. Experimental results demonstrate that, in groups of five fish, a white background and larger stimulus size significantly enhance guidance efficiency; however, performance markedly declines when group size increases to eight individuals.

Abstract

Guiding collective motion in biological groups is a fundamental challenge in understanding social interaction rules and developing automated systems for animal management. In this study, we propose a deep reinforcement learning (RL) framework for the closed-loop guidance of fish schools using virtual agents. These agents are controlled by policies trained via Proximal Policy Optimization (PPO) in simulation and deployed in physical experiments with rummy-nose tetras (Petitella bleheri), enabling real-time interaction between artificial agents and live individuals. To cope with the stochastic behavior of live individuals, we design a composite reward function to balance directional guidance with social cohesion. Our systematic evaluation of visual parameters shows that a white background and larger stimulus sizes maximize guidance efficacy in physical trials. Furthermore, evaluation across group sizes revealed that while the system demonstrates effective guidance for groups of five individuals, this capability markedly degrades as group size increases to eight. This study highlights the potential of deep RL for automated guidance of biological collectives and identifies challenges in maintaining artificial influence in larger groups.

Problem

• collective motion
• closed-loop guidance
• fish schools
• group size
• social cohesion

Innovation

• deep reinforcement learning
• closed-loop guidance
• virtual agents
• collective behavior
• Proximal Policy Optimization