One-Step Score-Based Density Ratio Estimation

2026-04-12

AI Summary

This work addresses the challenge in density ratio estimation where achieving both high accuracy and computational efficiency is difficult, particularly when the underlying distributions exhibit large discrepancies. To this end, the authors propose a partially analytical one-step estimation framework that decomposes the time score into spatial and temporal components. The temporal component is represented analytically using radial basis functions, transforming complex integrals into closed-form weighted sums and enabling high-precision estimation with only a single function evaluation. This approach uniquely combines the computational efficiency of direct methods with the accuracy of score-based techniques and establishes, for the first time, a theoretical error bound for temporal kernel approximation. Empirical results demonstrate that the method achieves a superior balance between accuracy and inference efficiency across multiple tasks, including density estimation, continuous KL divergence and mutual information estimation, and near out-of-distribution detection.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Density ratio estimation (DRE) is a useful tool for quantifying discrepancies between probability distributions, but existing approaches often involve a trade-off between estimation quality and computational efficiency. Classical direct DRE methods are usually efficient at inference time, yet their performance can seriously deteriorate when the discrepancy between distributions is large. In contrast, score-based DRE methods often yield more accurate estimates in such settings, but they typically require considerable repeated function evaluations and numerical integration. We propose One-step Score-based Density Ratio Estimation (OS-DRE), a partly analytic and solver-free framework designed to combine these complementary advantages. OS-DRE decomposes the time score into spatial and temporal components, representing the latter with an analytic radial basis function (RBF) frame. This formulation converts the otherwise intractable temporal integral into a closed-form weighted sum, thereby removing the need for numerical solvers and enabling DRE with only one function evaluation. We further analyze approximation conditions for the analytic frame, and establish approximation error bounds for both finitely and infinitely smooth temporal kernels, grounding the framework in existing approximation theory. Experiments across density estimation, continual Kullback-Leibler and mutual information estimation, and near out-of-distribution detection demonstrate that OS-DRE offers a favorable balance between estimation quality and inference efficiency.

Problem

• Density Ratio Estimation
• Score-based Methods
• Computational Efficiency
• Estimation Accuracy
• Numerical Integration

Innovation

• density ratio estimation
• score-based modeling
• analytic integration
• radial basis function
• one-step inference