Learning to Comparison-Shop

2025-11-10

Proceedings of the 34th ACM International Conference on Information and Knowledge Management

AI Summary

Traditional search ranking models for online marketplaces (e.g., Airbnb) evaluate listings in isolation, neglecting users’ prevalent comparative shopping behavior—where multiple listings are evaluated jointly within a session. Method: This paper proposes the first ranking framework that explicitly models price-comparison behavior by incorporating multi-item comparison intent into learning-to-rank. It jointly encodes list-level contextual signals and comparative decision processes via a deep neural network, enabling end-to-end optimization. Contribution/Results: The key innovation is the formalization of human comparative decision-making as learnable, differentiable ranking constraints—thereby aligning ranked outputs with actual user shopping behavior. Offline evaluation shows a 1.7% improvement in NDCG; online A/B testing demonstrates a 0.6% absolute increase in booking conversion rate—outperforming state-of-the-art baselines.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Uses of LLMs and GenAI for marketplace design, bidding, and strategic interactions

Abstract

In online marketplaces like Airbnb, users frequently engage in comparison shopping before making purchase decisions. Despite the prevalence of this behavior, a significant disconnect persists between mainstream e-commerce search engines and users' comparison needs. Traditional ranking models often evaluate items in isolation, disregarding the context in which users compare multiple items on a search results page. While recent advances in deep learning have sought to improve ranking accuracy, diversity, and fairness by encoding listwise context, the challenge of aligning search rankings with user comparison shopping behavior remains inadequately addressed. In this paper, we propose a novel ranking architecture - Learning-to-Comparison-Shop (LTCS) System - that explicitly models and learns users' comparison shopping behaviors. Through extensive offline and online experiments, we demonstrate that our approach yields statistically significant gains in key business metrics - improving NDCG by 1.7% and boosting booking conversion rate by 0.6% in A/B testing - while also enhancing user experience. We also compare our model against state-of-the-art approaches and demonstrate that LTCS significantly outperforms them.

Problem

• Addresses disconnect between e-commerce search engines and user comparison shopping needs.
• Models user comparison behaviors to improve ranking beyond isolated item evaluation.
• Enhances ranking accuracy and user experience through Learning-to-Comparison-Shop architecture.

Innovation

• Models user comparison shopping behaviors explicitly
• Proposes Learning-to-Comparison-Shop ranking architecture
• Improves ranking accuracy and booking conversion rates