Density estimation with atoms, and functional estimation for mixed discrete-continuous data

2025-08-03

AI Summary

Classical density estimation assumes absolute continuity of the underlying distribution, rendering it ill-suited for mixed discrete-continuous data and leading to theoretical inconsistency and degraded empirical performance. To address this, we propose a general, plug-and-play correction framework that automatically separates discrete and continuous components by identifying duplicate versus unique observations—achieving consistent estimation without additional assumptions, even when the discrete support and mixing proportion are unknown. Our method encapsulates any existing continuous density estimator as a black box, requiring no internal modifications and remaining fully compatible with classical nonparametric functional estimation theory. Theoretically, its convergence rate matches that of the base estimator up to logarithmic factors; empirically, it significantly improves estimation accuracy in both mixed and purely continuous settings, while incurring no performance loss on purely discrete or purely continuous data. The core innovation lies in replacing the conventional continuity assumption with a data-driven classification mechanism, thereby unifying density and functional estimation for mixed distributions.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization
• Planning, Routing, and Scheduling: Mixed Discrete/Continuous Planning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

In classical density (or density-functional) estimation, it is standard to assume that the underlying distribution has a density with respect to the Lebesgue measure. However, when the data distribution is a mixture of continuous and discrete components, the resulting methods are inconsistent in theory and perform poorly in practice. In this paper, we point out that a minor modification of existing methods for nonparametric density (functional) estimation can allow us to fully remove this assumption while retaining nearly identical theoretical guarantees and improved empirical performance. Our approach is very simple: data points that appear exactly once are likely to originate from the continuous component, whereas repeated observations are indicative of the discrete part. Leveraging this observation, we modify existing estimators for a broad class of functionals of the continuous component of the mixture; this modification is a "wrapper" in the sense that the user can use any underlying method of their choice for continuous density functional estimation. Our modifications deliver consistency without requiring knowledge of the discrete support, the mixing proportion, and without imposing additional assumptions beyond those needed in the absence of the discrete part. Thus, various theorems and existing software packages can be made automatically more robust, with absolutely no additional price when the data is not truly mixed.

Problem

• Estimating density for mixed discrete-continuous data
• Improving consistency in density-functional estimation
• Modifying nonparametric methods for robust performance

Innovation

• Modify existing nonparametric density estimation methods
• Identify continuous component via unique data points
• Wrapper approach for robust functional estimation