Fisher information flow in artificial neural networks

2025-09-02

AI Summary

This study investigates how artificial neural networks (ANNs) internally encode and transmit Fisher information during parameter estimation. We propose an information-aware training framework based on layer-wise monitoring of Fisher information flow. Through theoretical modeling and empirical analysis, we characterize the decay of parameter correlation information along both forward and backward propagation paths. Leveraging this insight, we devise a validation-free, model-agnostic early-stopping criterion: optimal estimation coincides with the peak of Fisher information transmission, while overfitting induces substantial information loss. This criterion provides an interpretable, quantifiable metric for training dynamics. Evaluated on experimental imaging physics data, our method enables real-time monitoring of training progression, significantly improving parameter estimation accuracy and model interpretability. The approach establishes a novel paradigm for AI-driven precision measurement systems.

Technology Category

• Cognitive Modeling & Cognitive Systems: Neural Spike Coding
• Machine Learning: Information Theory
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

The estimation of continuous parameters from measured data plays a central role in many fields of physics. A key tool in understanding and improving such estimation processes is the concept of Fisher information, which quantifies how information about unknown parameters propagates through a physical system and determines the ultimate limits of precision. With Artificial Neural Networks (ANNs) gradually becoming an integral part of many measurement systems, it is essential to understand how they process and transmit parameter-relevant information internally. Here, we present a method to monitor the flow of Fisher information through an ANN performing a parameter estimation task, tracking it from the input to the output layer. We show that optimal estimation performance corresponds to the maximal transmission of Fisher information, and that training beyond this point results in information loss due to overfitting. This provides a model-free stopping criterion for network training-eliminating the need for a separate validation dataset. To demonstrate the practical relevance of our approach, we apply it to a network trained on data from an imaging experiment, highlighting its effectiveness in a realistic physical setting.

Problem

• Tracking Fisher information flow in neural networks
• Understanding information loss due to overfitting
• Providing model-free stopping criterion for training

Innovation

• Monitoring Fisher information flow in ANNs
• Optimal estimation corresponds to maximal Fisher transmission
• Model-free training stopping criterion without validation data