Neural Prior Estimation: Learning Class Priors from Latent Representations

2026-02-19

AI Summary

This work addresses the systematic prediction bias in deep neural networks caused by skewed class priors under categorical imbalance. The authors propose the Neural Prior Estimator (NPE), which learns feature-conditioned log-priors in the latent representation space and is jointly trained with the backbone network, enabling adaptive prior correction without explicit class counts or distribution-dependent hyperparameters. Grounded in Neural Collapse theory, NPE recovers the true class log-priors up to an additive constant and integrates with logit adjustment to form the NPE-LA mechanism. Experiments demonstrate that NPE significantly improves overall performance on long-tailed CIFAR benchmarks and imbalanced semantic segmentation datasets (STARE and ADE20K), with notable gains in minority-class recognition.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Learning & Optimization for NLP
• Computer Vision: Learning & Optimization for CV

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Class imbalance induces systematic bias in deep neural networks by imposing a skewed effective class prior. This work introduces the Neural Prior Estimator (NPE), a framework that learns feature-conditioned log-prior estimates from latent representations. NPE employs one or more Prior Estimation Modules trained jointly with the backbone via a one-way logistic loss. Under the Neural Collapse regime, NPE is analytically shown to recover the class log-prior up to an additive constant, providing a theoretically grounded adaptive signal without requiring explicit class counts or distribution-specific hyperparameters. The learned estimate is incorporated into logit adjustment, forming NPE-LA, a principled mechanism for bias-aware prediction. Experiments on long-tailed CIFAR and imbalanced semantic segmentation benchmarks (STARE, ADE20K) demonstrate consistent improvements, particularly for underrepresented classes. NPE thus offers a lightweight and theoretically justified approach to learned prior estimation and imbalance-aware prediction.

Problem

• class imbalance
• class prior
• neural networks
• bias correction
• latent representations

Innovation

• Neural Prior Estimation
• Class Imbalance
• Logit Adjustment
• Latent Representations
• Neural Collapse