Anchor-based Maximum Discrepancy for Relative Similarity Testing

2025-10-12

AI Summary

Relative similarity testing aims to determine which of two distributions, $P$ or $Q$, is closer to a reference distribution $U$. Existing kernel-based methods rely on fixed kernels and pre-specified alternatives (e.g., “$Q$ is closer than $P$”), coupling kernel selection with hypothesis specification—leading to statistical bias, as any alternative admits a kernel that renders the test inconsistent. Method: We propose an end-to-end deep kernel learning framework that jointly optimizes hypothesis inference and kernel learning. We introduce the Anchored Maximum Discrepancy (AMD) criterion to simultaneously learn both the optimal kernel in a deep kernel space and the implicit relative similarity hypothesis. A unified two-stage testing procedure eliminates subjective hypothesis specification. Contribution/Results: We establish statistical consistency of the proposed test. Experiments across multiple benchmarks demonstrate significant improvements in detection power and robustness over state-of-the-art methods. The implementation is publicly available.

Technology Category

• Machine Learning: Kernel Methods
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The relative similarity testing aims to determine which of the distributions, P or Q, is closer to an anchor distribution U. Existing kernel-based approaches often test the relative similarity with a fixed kernel in a manually specified alternative hypothesis, e.g., Q is closer to U than P. Although kernel selection is known to be important to kernel-based testing methods, the manually specified hypothesis poses a significant challenge for kernel selection in relative similarity testing: Once the hypothesis is specified first, we can always find a kernel such that the hypothesis is rejected. This challenge makes relative similarity testing ill-defined when we want to select a good kernel after the hypothesis is specified. In this paper, we cope with this challenge via learning a proper hypothesis and a kernel simultaneously, instead of learning a kernel after manually specifying the hypothesis. We propose an anchor-based maximum discrepancy (AMD), which defines the relative similarity as the maximum discrepancy between the distances of (U, P) and (U, Q) in a space of deep kernels. Based on AMD, our testing incorporates two phases. In Phase I, we estimate the AMD over the deep kernel space and infer the potential hypothesis. In Phase II, we assess the statistical significance of the potential hypothesis, where we propose a unified testing framework to derive thresholds for tests over different possible hypotheses from Phase I. Lastly, we validate our method theoretically and demonstrate its effectiveness via extensive experiments on benchmark datasets. Codes are publicly available at: https://github.com/zhijianzhouml/AMD.

Problem

• Addresses kernel selection challenges in relative similarity testing
• Learns hypothesis and kernel simultaneously for robust testing
• Proposes anchor-based maximum discrepancy for distribution comparison

Innovation

• Simultaneously learns hypothesis and kernel
• Uses deep kernel space for maximum discrepancy
• Two-phase testing with unified significance assessment