Density Ratio Estimation with Conditional Probability Paths

2025-02-04

AI Summary

To address the high computational cost and low accuracy of time-score estimation in high-dimensional density-ratio estimation, this paper proposes a novel conditional time-score estimation framework. The method reformulates density-ratio estimation as integration of the time score along a probability path and introduces a closed-form loss function amenable to analytical optimization. Its core contribution is the first introduction of the “conditional time-score estimation” paradigm, which leverages conditioning variables to enable efficient and stable learning. Theoretical analysis establishes an upper bound on the density-ratio estimation error. Experiments demonstrate that the proposed method significantly accelerates training and achieves state-of-the-art or leading accuracy across multiple high-dimensional benchmark tasks.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Search and Optimization: Learning to Search
• Intelligent Robots: State Estimation

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Density ratio estimation in high dimensions can be reframed as integrating a certain quantity, the time score, over probability paths which interpolate between the two densities. In practice, the time score has to be estimated based on samples from the two densities. However, existing methods for this problem remain computationally expensive and can yield inaccurate estimates. Inspired by recent advances in generative modeling, we introduce a novel framework for time score estimation, based on a conditioning variable. Choosing the conditioning variable judiciously enables a closed-form objective function. We demonstrate that, compared to previous approaches, our approach results in faster learning of the time score and competitive or better estimation accuracies of the density ratio on challenging tasks. Furthermore, we establish theoretical guarantees on the error of the estimated density ratio.

Problem

• Estimating high-dimensional density ratios efficiently
• Improving accuracy in time score estimation
• Developing a novel framework with theoretical guarantees

Innovation

• Conditioning variable for estimation
• Closed-form objective function
• Faster, accurate density ratio estimation