CINDES: Classification induced neural density estimator and simulator

2025-09-30

AI Summary

Existing neural density estimation methods exhibit strong empirical performance but struggle to simultaneously enforce non-negativity and unit-mass constraints, and lack theoretical guarantees for adaptive convergence on low-dimensional structured densities. This paper proposes a structure-agnostic, classification-induced neural density estimation framework: by reformulating density ratio estimation as a binary classification task, it implicitly satisfies probabilistic constraints without explicit regularization. We establish theoretical guarantees showing that, under high-dimensional densities supported on low-dimensional manifolds, the estimator achieves adaptive convergence rates strictly faster than standard nonparametric rates. Moreover, the framework naturally yields an efficient sampling mechanism. Extensive experiments on synthetic and real-world datasets demonstrate significant improvements in convergence speed, density estimation accuracy, and sample quality. To our knowledge, this work provides the first rigorous theoretical connection between the classification paradigm and neural density estimation.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Non-convex Optimization
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Neural network-based methods for (un)conditional density estimation have recently gained substantial attention, as various neural density estimators have outperformed classical approaches in real-data experiments. Despite these empirical successes, implementation can be challenging due to the need to ensure non-negativity and unit-mass constraints, and theoretical understanding remains limited. In particular, it is unclear whether such estimators can adaptively achieve faster convergence rates when the underlying density exhibits a low-dimensional structure. This paper addresses these gaps by proposing a structure-agnostic neural density estimator that is (i) straightforward to implement and (ii) provably adaptive, attaining faster rates when the true density admits a low-dimensional composition structure. Another key contribution of our work is to show that the proposed estimator integrates naturally into generative sampling pipelines, most notably score-based diffusion models, where it achieves provably faster convergence when the underlying density is structured. We validate its performance through extensive simulations and a real-data application.

Problem

• Proposes a neural density estimator with easy implementation and theoretical guarantees
• Achieves adaptive convergence rates for low-dimensional structured densities
• Integrates with generative sampling pipelines like diffusion models for faster convergence

Innovation

• Classification induced neural density estimator design
• Structure-agnostic implementation with proven adaptability
• Integration with score-based diffusion generative models