FANoise: Singular Value-Adaptive Noise Modulation for Robust Multimodal Representation Learning

2025-11-25

AI Summary

To address insufficient robustness in multimodal representation learning, existing methods typically rely on static or heuristic noise injection, neglecting the dynamic evolution of feature distributions. This paper proposes FANoise, an adaptive noise injection framework grounded in dual perspectives—gradient dynamics and feature distribution statistics. Its core innovation is a singular-value-adaptive mechanism that dynamically modulates noise intensity according to the spectral properties of encoder features, thereby enhancing regularization while preserving training stability. Integrated within the InfoNCE contrastive learning framework, FANoise enables data-driven noise modulation. Extensive experiments across multiple vision-language models demonstrate that FANoise significantly improves generalization performance on cross-modal retrieval and understanding tasks. The method exhibits strong cross-architectural applicability and offers theoretical interpretability through its principled, spectrum-aware design.

Technology Category

• Machine Learning: Multimodal Learning
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Representation learning is fundamental to modern machine learning, powering applications such as text retrieval and multimodal understanding. However, learning robust and generalizable representations remains challenging. While prior work has demonstrated that active noise injection, a form of data augmentation, can enhance encoding performance, most existing methods rely on heuristic or static noise, overlooking the dynamic nature of feature distributions during training. In this work, we systematically study the role of noise in representation learning from both gradient-based and feature distribution perspectives, using InfoNCE loss as a representative example. Focusing on multimodal representation learning, we propose FANoise, a novel feature-adaptive noise injection strategy. By leveraging the dynamics of contrastive learning, FANoise effectively mitigates the negative impacts of noise while preserving its benefits. Under this theoretically grounded framework, comprehensive experiments demonstrate that FANoise consistently improves overall performance on multimodal tasks across various base VLM models.

Problem

• Develops adaptive noise injection for robust multimodal representation learning
• Addresses static noise limitations in dynamic feature distributions during training
• Enhances performance across various vision-language models with theoretical framework

Innovation

• Feature-adaptive noise injection strategy
• Leverages contrastive learning dynamics
• Improves multimodal task performance