Stream-aware Side Adaptation for Large Pre-trained Multimodal Embedding Models in Sequential Recommendation

2026-07-12

AI Summary

This work addresses the performance limitations of large pretrained multimodal embedding models in sequential recommendation due to domain shift and the degradation of existing side-tuning methods with increasing network depth. To overcome these challenges, the authors propose Stresa, a framework that operates with a frozen backbone and introduces Stream-aware Hidden Adapter Fusion (SHAF) to preserve historical side memory. Additionally, Stresa incorporates Residual Stream Adapters (ReSA) for selective cross-layer residual updates, complemented by a progressive Sigmoid fusion mechanism. Extensive experiments demonstrate that Stresa consistently outperforms current side-tuning approaches and state-of-the-art baselines across multiple public datasets and diverse backbone architectures, significantly enhancing recommendation performance.

Abstract

Recently, large pretrained multimodal embedding models such as Qwen3-VL Embedding have shown strong promise for sequential recommendation, as they provide reusable semantic item representations across modalities and domains. However, directly using these embeddings often leads to suboptimal performance because of domain misalignment. Efficient side adaptation is therefore an attractive solution. Although adapting all backbone layers should help, existing side adapters often degrade with depth, prompting layer dropping despite the loss of useful hidden states. This is due to two major challenges: (1) the lack of modeling in selecting fused representations during residual addition, and (2) the insufficient preservation of earlier representations during progressive sigmoid fusion. This paper therefore asks a practical question: How can we design a side adaptation approach that effectively unlocks the potential of large pre-trained multimodal embedding models? To address this question, we propose Stresa, a stream-aware side-adaptation framework for frozen large pre-trained multimodal embedding models in sequential recommendation. Stresa introduces Stream-aware Hidden-Adapter Fusion (SHAF) to preserve historical side memory during fusion and Residual Stream Adapter (ReSA) to produce selective residual updates across layers. Empirically, Stresa consistently outperforms standard side adapters and state-of-the-art baselines on public datasets across multiple backbone embedding models. These results highlight the promise of adapting large embedding models for sequential recommendation. Our code is publicly available at https://github.com/GAIR-Lab/Stresa.

Problem

• sequential recommendation
• multimodal embedding
• side adaptation
• domain misalignment
• large pre-trained models

Innovation

• side adaptation
• multimodal embedding
• sequential recommendation
• stream-aware fusion
• frozen backbone