TeMo: Temperature Modulation for Multimodal Contrastive Learning

2026-09-07

AI Summary

本文提出TeMo框架，通过自适应调整每对正负样本的温度参数以解决对比学习中固定或全局温度设置的问题，提高多模态对比学习的效果。

Abstract

Contrastive learning approaches achieve strong performance by training models to bring similar samples closer while pushing dissimilar samples apart. A crucial component of contrastive learning is the temperature hyperparameter $\tau$, which controls the penalty strength applied to negative samples. However, most existing methods either fix this hyperparameter or learn a global value during training. In this paper, we introduce TeMo, Temperature Modulation framework, a similarity-based modulation approach that adaptively adjusts the temperature for each positive-negative pair according to their similarity, enabling more fine-grained multimodal contrastive learning. Our approach seamlessly integrates temperature-modulated multimodal and unimodal losses with the standard multimodal contrastive loss by gradually transitioning between them. This design allows the model to capture both coarse- and fine-grained semantics at different training stages. Extensive experiments demonstrate that each component of TeMo consistently enhances performance across diverse zero-shot retrieval and classification tasks, establishing new state-of-the-art results.

Problem

• contrastive learning
• temperature hyperparameter
• multimodal contrastive learning
• similarity-based modulation

Innovation

• Temperature Modulation
• Adaptive Adjustment
• Multimodal Contrastive Learning
• Similarity-based