Avoiding Death through Fear Intrinsic Conditioning

2025-06-05

AI Summary

In reinforcement learning, agents struggle to avoid lethal states—such as “death”—in sparse-reward and partially observable settings due to the absence of explicit feedback. To address this, we propose an amygdala-inspired intrinsic reward mechanism, modeling fear acquisition from developmental neuroscience as a learnable internal signal. Implemented via a memory-augmented neural network (MANN), the mechanism enables autonomous lethal-state avoidance and supports adaptive modulation of fear thresholds, reproducing behavioral phenotypes across the generalized anxiety disorder spectrum. This work constitutes the first formalization of biological fear conditioning as a trainable intrinsic reward signal. Evaluated in the MiniWorld-Sidewalk POMDP—a high-risk, sparse-reward environment—the approach significantly improves agent survival rate and task completion rate, demonstrating the efficacy of biologically grounded intrinsic motivation for safety-critical decision-making under uncertainty.

Technology Category

• Multiagent Systems: Adversarial Agents
• Machine Learning: Reinforcement Learning
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Agentic search
• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications

Abstract

Biological and psychological concepts have inspired reinforcement learning algorithms to create new complex behaviors that expand agents' capacity. These behaviors can be seen in the rise of techniques like goal decomposition, curriculum, and intrinsic rewards, which have paved the way for these complex behaviors. One limitation in evaluating these methods is the requirement for engineered extrinsic for realistic environments. A central challenge in engineering the necessary reward function(s) comes from these environments containing states that carry high negative rewards, but provide no feedback to the agent. Death is one such stimuli that fails to provide direct feedback to the agent. In this work, we introduce an intrinsic reward function inspired by early amygdala development and produce this intrinsic reward through a novel memory-augmented neural network (MANN) architecture. We show how this intrinsic motivation serves to deter exploration of terminal states and results in avoidance behavior similar to fear conditioning observed in animals. Furthermore, we demonstrate how modifying a threshold where the fear response is active produces a range of behaviors that are described under the paradigm of general anxiety disorders (GADs). We demonstrate this behavior in the Miniworld Sidewalk environment, which provides a partially observable Markov decision process (POMDP) and a sparse reward with a non-descriptive terminal condition, i.e., death. In effect, this study results in a biologically-inspired neural architecture and framework for fear conditioning paradigms; we empirically demonstrate avoidance behavior in a constructed agent that is able to solve environments with non-descriptive terminal conditions.

Problem

• Develops fear-inspired intrinsic reward to avoid terminal states
• Addresses lack of feedback in high-negative-reward environments
• Proposes neural architecture mimicking amygdala-based fear conditioning

Innovation

• Intrinsic reward function inspired by amygdala development
• Memory-augmented neural network (MANN) architecture
• Fear conditioning paradigm for terminal state avoidance