🤖 AI Summary
Existing multimodal information bottleneck (MIB) methods suffer from two critical limitations: (1) employing a fixed, empirically chosen regularization weight lacking theoretical justification, and (2) ignoring the imbalance in task-relevant information across modalities, thereby hindering optimal MIB learning. To address these, we propose OMIB—a theoretically grounded framework that (i) derives the first analytically tractable feasibility bound for the regularization weight; (ii) introduces a modality-adaptive dynamic weighting mechanism to explicitly model and balance task-relevant information across modalities; and (iii) establishes a rigorous variational information-theoretic foundation ensuring provably convergent optimization to the optimal MIB solution. OMIB integrates variational information bottleneck optimization, modality-specific regularization, and principled information-theoretic modeling. We validate its theoretical properties on synthetic data and demonstrate significant improvements over state-of-the-art methods across multiple downstream multimodal tasks.
📝 Abstract
Leveraging high-quality joint representations from multimodal data can greatly enhance model performance in various machine-learning based applications. Recent multimodal learning methods, based on the multimodal information bottleneck (MIB) principle, aim to generate optimal MIB with maximal task-relevant information and minimal superfluous information via regularization. However, these methods often set ad hoc regularization weights and overlook imbalanced task-relevant information across modalities, limiting their ability to achieve optimal MIB. To address this gap, we propose a novel multimodal learning framework, Optimal Multimodal Information Bottleneck (OMIB), whose optimization objective guarantees the achievability of optimal MIB by setting the regularization weight within a theoretically derived bound. OMIB further addresses imbalanced task-relevant information by dynamically adjusting regularization weights per modality, promoting the inclusion of all task-relevant information. Moreover, we establish a solid information-theoretical foundation for OMIB's optimization and implement it under the variational approximation framework for computational efficiency. Finally, we empirically validate the OMIB's theoretical properties on synthetic data and demonstrate its superiority over the state-of-the-art benchmark methods in various downstream tasks.