Online activity prediction via generalized Indian buffet process models

2025-05-26

AI Summary

This work addresses the problem of real-time sparse counting prediction for user activities (first-time vs. repeated triggers) in A/B experiments. We propose a Bayesian nonparametric model grounded in the stable beta scaling process, which imposes no parametric distributional assumptions and naturally captures heavy-tailed, sparse, and dynamically evolving user behavior. All posterior and predictive distributions admit closed-form expressions, enabling rapid cold-start deployment (within 1–3 days) and principled uncertainty calibration. Our key contribution lies in integrating the generalized Indian Buffet Process with a stable beta-scaled prior, yielding an interpretable and computationally tractable nonparametric framework. In a retrospective evaluation across 1,774 live online experiments at Amazon, our method achieves statistically significant improvements over state-of-the-art approaches in both predictive accuracy and calibration for new-user counts and total trigger counts.

Technology Category

• Reasoning under Uncertainty: Relational Probabilistic Models
• Machine Learning: Bayesian Learning
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Online A/B experiments generate millions of user-activity records each day, yet experimenters need timely forecasts to guide roll-outs and safeguard user experience. Motivated by the problem of activity prediction for A/B tests at Amazon, we introduce a Bayesian nonparametric model for predicting both first-time and repeat triggers in web experiments. The model is based on the stable beta-scaled process prior, which allows for capturing heavy-tailed behaviour without strict parametric assumptions. All posterior and predictive quantities are available in closed form, allowing for fast inference even on large-scale datasets. Simulation studies and a retrospective analysis of 1,774 production experiments show improved accuracy in forecasting new users and total triggers compared with state-of-the-art competitors, especially when only a few pilot days are observed. The framework enables shorter tests while preserving calibrated uncertainty estimates. Although motivated by Amazon's experimentation platform, the method extends to other applications that require rapid, distribution-free prediction of sparse count processes.

Problem

• Predicting user activity in online A/B tests
• Modeling first-time and repeat triggers accurately
• Enabling rapid forecasting with sparse data

Innovation

• Bayesian nonparametric model for activity prediction
• Stable beta-scaled process prior for heavy-tailed behavior
• Closed-form posterior and predictive for fast inference