🤖 AI Summary
This work addresses the limited interpretability of cosine similarity in pretrained embedding spaces, where absolute similarity values are concentrated in a narrow range due to anisotropy. The authors propose a monotonic calibration method based on isotonic regression that reparameterizes similarity scores without altering the underlying embeddings or their geometric structure. The approach strictly preserves the original ranking order of similarities, thereby maintaining all ordinal-dependent structures—such as nearest neighbors, angular rankings, and threshold graphs—intact. The calibrated similarities achieve near-perfect alignment in absolute values while retaining 98% local stability under seven types of perturbations and fully preserving rank correlation with the original similarities.
📝 Abstract
While raw cosine similarity in pretrained embedding spaces exhibits strong rank correlation with human judgments, anisotropy induces systematic miscalibration of absolute values: scores concentrate in a narrow high-similarity band regardless of actual semantic relatedness, limiting interpretability as a quantitative measure. Prior work addresses this by modifying the embedding space (whitening, contrastive fine tuning), but such transformations alter geometric structure and require recomputing all embeddings. Using isotonic regression trained on human similarity judgments, we construct a monotonic transformation that achieves near-perfect calibration while preserving rank correlation and local stability(98% across seven perturbation types). Our contribution is not to replace cosine similarity, but to restore interpretability of its absolute values through monotone calibration, without altering its ranking properties. We characterize isotonic calibration as an order-preserving reparameterization and prove that all order-based constructions (angular ordering, nearest neighbors, threshold graphs and quantile-based decisions) are invariant under this transformation.