Analyzing the Shopping Journey: Computing Shelf Browsing Visits in a Physical Retail Store

2026-01-02

arXiv.org

AI Summary

This study addresses the challenge of automatically recognizing in-store shelf browsing behavior by customers in physical retail environments. It proposes a machine vision approach leveraging overhead camera videos to extract 3D human trajectories and identify customer dwell events—termed “shelf visits”—in front of product displays. The method is calibrated using manually annotated data and validated across multiple store locations. Notably, this work presents the first model capable of effectively generalizing shelf-browsing recognition to unseen store environments. Furthermore, it establishes an analytical framework linking observed browsing patterns with actual purchase outcomes. Experimental results demonstrate that the proposed approach reliably detects cross-store browsing behaviors and reveals significant associations between specific browsing characteristics and subsequent purchasing decisions.

Technology Category

• Computer Vision: Image and Video Retrieval
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Motivated by recent challenges in the deployment of robots into customer-facing roles within retail, this work introduces a study of customer activity in physical stores as a step toward autonomous understanding of shopper intent. We introduce an algorithm that computes shoppers'``shelf visits''-- capturing their browsing behavior in the store. Shelf visits are extracted from trajectories obtained via machine vision-based 3D tracking and overhead cameras. We perform two independent calibrations of the shelf visit algorithm, using distinct sets of trajectories (consisting of 8138 and 15129 trajectories), collected in different stores and labeled by human reviewers. The calibrated models are then evaluated on trajectories held out of the calibration process both from the same store on which calibration was performed and from the other store. An analysis of the results shows that the algorithm can recognize customers'browsing activity when evaluated in an environment different from the one on which calibration was performed. We then use the model to analyze the customers'``browsing patterns''on a large set of trajectories and their relation to actual purchases in the stores. Finally, we discuss how shelf browsing information could be used for retail planning and in the domain of human-robot interaction scenarios.

Problem

• shelf visits
• customer browsing behavior
• retail analytics
• shopper intent
• physical retail store

Innovation

• shelf visit
• 3D trajectory tracking
• customer browsing behavior
• cross-store generalization
• retail analytics