Learning Distributions over Permutations and Rankings with Factorized Representations

2025-05-30

AI Summary

This work addresses the fundamental challenge of modeling probability distributions over permutations—a core problem in ranking, combinatorial optimization, and structured prediction. We propose an unconstrained, factorized deep learning framework that, for the first time, integrates the Lehmer code bijection and the Fisher–Yates sampling mechanism into neural networks, enabling differentiable parameterization of permutations via insertion vectors. The framework subsumes classical models (e.g., Mallows) and supports flexible modeling—from unimodal to multimodal permutation distributions. Our contributions are: (1) the first benchmark suite for permutation distribution learning, comprising jigsaw puzzle reconstruction, cyclic permutation recovery, and movie re-ranking tasks; and (2) empirical results showing consistent and significant improvements over state-of-the-art methods across all tasks—even the minimal instantiation learns non-degenerate, valid permutation distributions, whereas conventional approaches often fail catastrophically.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Learning distributions over permutations is a fundamental problem in machine learning, with applications in ranking, combinatorial optimization, structured prediction, and data association. Existing methods rely on mixtures of parametric families or neural networks with expensive variational inference procedures. In this work, we propose a novel approach that leverages alternative representations for permutations, including Lehmer codes, Fisher-Yates draws, and Insertion-Vectors. These representations form a bijection with the symmetric group, allowing for unconstrained learning using conventional deep learning techniques, and can represent any probability distribution over permutations. Our approach enables a trade-off between expressivity of the model family and computational requirements. In the least expressive and most computationally efficient case, our method subsumes previous families of well established probabilistic models over permutations, including Mallow's and the Repeated Insertion Model. Experiments indicate our method significantly outperforms current approaches on the jigsaw puzzle benchmark, a common task for permutation learning. However, we argue this benchmark is limited in its ability to assess learning probability distributions, as the target is a delta distribution (i.e., a single correct solution exists). We therefore propose two additional benchmarks: learning cyclic permutations and re-ranking movies based on user preference. We show that our method learns non-trivial distributions even in the least expressive mode, while traditional models fail to even generate valid permutations in this setting.

Problem

• Learning distributions over permutations efficiently
• Representing permutations using alternative factorized methods
• Evaluating models beyond delta distribution benchmarks

Innovation

• Uses Lehmer codes for permutation representation
• Leverages Fisher-Yates draws for learning
• Employs Insertion-Vectors for unconstrained learning