RetailSMV: Exocentric vs. Egocentric Adaptation of Foundation Video World Models in Retail

2026-06-30

AI Summary

Existing general-purpose video world models struggle to adapt to real-world retail environments. This work addresses this gap by introducing RetailSMV, the first synchronized multi-view retail video dataset centered on clerk activities, and conducts a systematic evaluation of egocentric, exocentric, and fused viewpoints through parameter-efficient fine-tuning of the Cosmos3-Nano model using LoRA. Experimental results reveal that models trained solely on exocentric views match or outperform fused-view configurations across most metrics, achieving significantly better performance in LPIPS, PSNR, and DreamSim. Furthermore, incorporating exocentric data enhances egocentric model performance, whereas adding egocentric data degrades exocentric model accuracy—challenging the common assumption that multi-view fusion inherently yields superior results.

Abstract

Foundation video diffusion models are increasingly viewed as world simulators for embodied agents, yet their pretraining on internet-scale generic video leaves them poorly aligned with real-world deployment domains. We study parameter-efficient adaptation of a pretrained foundation video world model to retail scenes: when synchronized egocentric and exocentric video of the same activity are available, which viewpoint of training data produces the strongest adapted model? We introduce RetailSMV (Retail Synchronized Multi-View), a corpus of 32,105 captioned retail clips from five supermarkets with synchronized ego/exo capture from the store-staff perspective (stocking, arranging, weighing, managing supply carts, scanning at checkout), rather than the customer-centric framing of prior retail video corpora, and train three matched Low-Rank Adaptation (LoRA) configurations of Cosmos3-Nano (egocentric-only, exocentric-only, combined) under identical hyperparameters. On a 200-clip held-out test set evaluated with seven complementary metrics under a strict paired statistical protocol, exocentric-only adaptation matches or exceeds combined adaptation on six of seven point estimates and is significantly better on LPIPS, PSNR, and DreamSim, despite training on only 15,985 exocentric clips (versus 32,105 for combined). A symmetric paired comparison further shows that adding exocentric data to egocentric-only training helps while adding egocentric data to exocentric-only training hurts. The absolute adaptation gap is largest at the shortest rollout time, identifying the near-horizon prediction window as the regime in which adaptation is most beneficial.

Problem

• video world models
• retail adaptation
• egocentric vs exocentric
• foundation models
• parameter-efficient adaptation

Innovation

• video world models
• parameter-efficient adaptation
• egocentric vs exocentric
• retail video dataset
• LoRA