Don't Forget Its Variance! The Minimum Path Variance Principle for Accurate and Stable Score-Based Density Ratio Estimation

2026-01-31

AI Summary

This work addresses the critical yet theoretically underexplored issue that path selection significantly affects performance in score-based density ratio estimation. To resolve this, we propose the Minimum Path Variance (MinPV) principle, which explicitly models and minimizes the variance of time-dependent scores along the path to optimize the objective function. We derive, for the first time, a closed-form expression for path variance, transforming the previously intractable variance minimization problem into an optimizable form. Furthermore, we introduce a data-adaptive path parameterization based on a mixture of Kumaraswamy distributions, enabling end-to-end training. The resulting method achieves state-of-the-art estimation accuracy and stability across multiple challenging benchmarks.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Score-based methods have emerged as a powerful framework for density ratio estimation (DRE), but they face an important paradox in that, while theoretically path-independent, their practical performance depends critically on the chosen path schedule. We resolve this issue by proving that tractable training objectives differ from the ideal, ground-truth objective by a crucial, overlooked term: the path variance of the time score. To address this, we propose MinPV (\textbf{Min}imum \textbf{P}ath \textbf{V}ariance) Principle, which introduces a principled heuristic to minimize the overlooked path variance. Our key contribution is the derivation of a closed-form expression for the variance, turning an intractable problem into a tractable optimization. By parameterizing the path with a flexible Kumaraswamy Mixture Model, our method learns a data-adaptive, low-variance path without heuristic selection. This principled optimization of the complete objective yields more accurate and stable estimators, establishing new state-of-the-art results on challenging benchmarks.

Problem

• density ratio estimation
• score-based methods
• path variance
• path dependence
• estimation stability

Innovation

• score-based density ratio estimation
• path variance
• MinPV principle
• closed-form variance
• Kumaraswamy Mixture Model