Minimal Sufficient Representations for Self-interpretable Deep Neural Networks

2026-03-25

AI Summary

This work proposes DeepIn, a novel framework that addresses the poor interpretability of over-parameterized deep neural networks and the challenge of identifying minimal sufficient architectures. By integrating minimal sufficient representation learning with statistical inference, DeepIn simultaneously performs variable selection and network pruning through adaptive representation learning, non-asymptotic error analysis, and hypothesis testing. The resulting model retains the expressive power of standard DNNs while offering statistical rigor and self-explainability. Evaluated on biomedical and visual tasks, DeepIn reduces prediction error by up to 30% and automatically uncovers human-interpretable discriminative patterns, substantially enhancing both model interpretability and generalization performance.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Representation Learning for Vision
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Deep neural networks (DNNs) achieve remarkable predictive performance but remain difficult to interpret, largely due to overparameterization that obscures the minimal structure required for interpretation. Here we introduce DeepIn, a self-interpretable neural network framework that adaptively identifies and learns the minimal representation necessary for preserving the full expressive capacity of standard DNNs. We show that DeepIn can correctly identify the minimal representation dimension, select relevant variables, and recover the minimal sufficient network architecture for prediction. The resulting estimator achieves optimal non-asymptotic error rates that adapt to the learned minimal dimension, demonstrating that recovering minimal sufficient structure fundamentally improves generalization error. Building on these guarantees, we further develop hypothesis testing procedures for both selected variables and learned representations, bridging deep representation learning with formal statistical inference. Across biomedical and vision benchmarks, DeepIn improves both predictive accuracy and interpretability, reducing error by up to 30% on real-world datasets while automatically uncovering human-interpretable discriminative patterns. Our results suggest that interpretability and statistical rigor can be embedded directly into deep architectures without sacrificing performance.

Problem

• interpretability
• minimal sufficient representation
• overparameterization
• deep neural networks
• statistical inference

Innovation

• minimal sufficient representation
• self-interpretable neural networks
• adaptive dimensionality reduction
• statistical inference in deep learning
• non-asymptotic error rates