Sequential Modality Dropout for Robust Multi-Modal Sequential Recommendation

2026-08-10

AI Summary

This work addresses the performance degradation of multimodal sequential recommendation models under real-world deployment scenarios where item modalities—such as images or text—are often missing, despite being assumed complete during training. To enhance robustness, the authors propose Sequential Modality Dropout (SMD), a novel training strategy that randomly masks an entire modality across a user’s interaction sequence with probability \( p \). SMD introduces, for the first time in sequential recommendation, a modality-level dropout mechanism spanning users’ historical interactions and optionally incorporates cross-modal reconstruction loss to encourage modality-agnostic sequential representations. Compatible with mainstream architectures like MM-SASRec and IISAN, SMD significantly improves robustness on four Amazon datasets: under text absence, accuracy retention increases by 1.0–3.2×; even with 95% modality missingness, HR@10 retention reaches 61% (versus 22% for baselines), while preserving near-original performance when all modalities are present.

Abstract

Multi-modal sequential recommenders assume every item carries every modality, but real product catalogs often miss images or text, and a model trained on complete data loses much of its recommendation accuracy when a modality is unavailable at serving time. We propose Sequential Modality Dropout (SMD): during training, each modality stream (image and text) is independently erased with probability p for an entire user interaction history, so the model learns to predict the next item without relying on any single modality. We measure robustness by retention, the fraction of a model's full-modality accuracy (HR@10) that survives when a modality is removed at test time. Across four backbones (MM-SASRec, IISAN, MISSRec, and fMRLRec) on four Amazon domains, SMD raises text retention by 1.0 to 3.2x at essentially no cost to full-modality accuracy; under an extreme 95% per-item missing rate, it retains 61% of HR@10 versus 22% without (a 2.8x improvement). An optional cross-modal reconstruction loss further lifts retention from 90% to 98% on a simple additive backbone under severe text missingness. SMD is a four-line, architecture-agnostic change that makes multi-modal sequential recommenders robust to the missing modalities they actually encounter in deployment.

Problem

• multi-modal sequential recommendation
• missing modality
• robustness
• modality dropout
• recommendation accuracy

Innovation

• Sequential Modality Dropout
• multi-modal sequential recommendation
• modality robustness
• missing modality
• cross-modal reconstruction