Distributional Evaluation of Generative Models via Relative Density Ratio

2025-10-29

AI Summary

This work addresses longstanding challenges in generative model evaluation—namely, the difficulty of quantifying distributional discrepancies, poor interpretability, and numerical instability. We propose a novel evaluation framework grounded in Relative Density Ratio (RDR), the first to treat RDR as a functional evaluation metric. Our approach inherits the theoretical properties of φ-divergences and enables both sample-level quality diagnostics and feature-level attribution analysis. To estimate the RDR function, we formulate a convex variational objective derived from the φ-divergence representation, and rigorously establish its statistical reliability and numerical stability via M-estimation theory and convergence analysis of neural networks in anisotropic Besov spaces. Experiments on MNIST, CelebA64, and microbiome datasets demonstrate that our method accurately discriminates model performance across support coverage, mode fidelity, and overlap regions—substantially enhancing both interpretability and discriminative power in generative evaluation.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

We propose a functional evaluation metric for generative models based on the relative density ratio (RDR) designed to characterize distributional differences between real and generated samples. We show that the RDR as a functional summary of the goodness-of-fit for the generative model, possesses several desirable theoretical properties. It preserves $φ$-divergence between two distributions, enables sample-level evaluation that facilitates downstream investigations of feature-specific distributional differences, and has a bounded range that affords clear interpretability and numerical stability. Functional estimation of the RDR is achieved efficiently through convex optimization on the variational form of $φ$-divergence. We provide theoretical convergence rate guarantees for general estimators based on M-estimator theory, as well as the convergence rates of neural network-based estimators when the true ratio is in the anisotropic Besov space. We demonstrate the power of the proposed RDR-based evaluation through numerical experiments on MNIST, CelebA64, and the American Gut project microbiome data. We show that the estimated RDR not only allows for an effective comparison of the overall performance of competing generative models, but it can also offer a convenient means of revealing the nature of the underlying goodness-of-fit. This enables one to assess support overlap, coverage, and fidelity while pinpointing regions of the sample space where generators concentrate and revealing the features that drive the most salient distributional differences.

Problem

• Proposes a functional metric using relative density ratio for generative model evaluation
• Enables sample-level analysis of feature-specific distributional differences between datasets
• Provides theoretical guarantees and interpretable comparison of model performance characteristics

Innovation

• Relative density ratio measures generative model differences
• Convex optimization estimates ratio via variational divergence
• Neural networks approximate ratio with theoretical guarantees