Experimental Assortments for Choice Estimation and Nest Identification

2026-02-17

AI Summary

This study addresses the problem of efficiently collecting choice data using a minimal number of assortments to estimate choice models over $n$ products and to automatically uncover the latent “nest” structure in nested logit models. The authors propose a structured, non-adaptive experimental design that requires only $O(\log n)$ assortments to effectively estimate a broad class of choice models. They further introduce a novel algorithm that, for the first time, consistently estimates nested logit models without requiring prior knowledge of the nest structure, guaranteeing correct recovery of nests under arbitrary true choice distributions. Empirical validation on 70 million user records from the Dream11 platform demonstrates that the inferred nests significantly outperform those derived from feature-based clustering, yielding higher out-of-sample prediction accuracy and meaningful business interpretability.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search
• Game Theory and Economic Paradigms: Social Choice / Voting

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

What assortments (subsets of items) should be offered, to collect data for estimating a choice model over $n$ total items? We propose a structured, non-adaptive experiment design requiring only $O(\log n)$ distinct assortments, each offered repeatedly, that consistently outperforms randomized and other heuristic designs across an extensive numerical benchmark that estimates multiple different choice models under a variety of (possibly mis-specified) ground truths. We then focus on Nested Logit choice models, which cluster items into "nests" of close substitutes. Whereas existing Nested Logit estimation procedures assume the nests to be known and fixed, we present a new algorithm to identify nests based on collected data, which when used in conjunction with our experiment design, guarantees correct identification of nests under any Nested Logit ground truth. Our experiment design was deployed to collect data from over 70 million users at Dream11, an Indian fantasy sports platform that offers different types of betting contests, with rich substitution patterns between them. We identify nests based on the collected data, which lead to better out-of-sample choice prediction than ex-ante clustering from contest features. Our identified nests are ex-post justifiable to Dream11 management.

Problem

• assortment design
• choice model estimation
• nest identification
• Nested Logit
• experiment design

Innovation

• experiment design
• choice modeling
• Nested Logit
• nest identification
• assortment optimization