A Survey on Mechanistic Interpretability for Multi-Modal Foundation Models

2025-02-22

AI Summary

Multimodal foundation models (MMFMs) suffer from insufficient mechanistic interpretability, and their fundamental differences from unimodal large language models (LLMs) remain poorly understood. Method: We propose the first structured taxonomy of MMFM interpretability methods, systematically integrating attribution analysis, feature disentanglement, neuron activation tracing, concept activation vectors (CAVs), and module-level interventions across contrastive, generative, and text-to-image architectures. Contribution/Results: Our analysis uncovers core mechanistic disparities in cross-modal interaction—particularly in representation alignment, information bottlenecks, and gradient propagation—distinguishing MMFMs from unimodal LLMs. We identify critical bottlenecks in current MMFM interpretability and introduce the first unified evaluation framework. This work provides both theoretical foundations and practical guidelines for designing, diagnosing, and optimizing trustworthy multimodal AI systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

The rise of foundation models has transformed machine learning research, prompting efforts to uncover their inner workings and develop more efficient and reliable applications for better control. While significant progress has been made in interpreting Large Language Models (LLMs), multimodal foundation models (MMFMs) - such as contrastive vision-language models, generative vision-language models, and text-to-image models - pose unique interpretability challenges beyond unimodal frameworks. Despite initial studies, a substantial gap remains between the interpretability of LLMs and MMFMs. This survey explores two key aspects: (1) the adaptation of LLM interpretability methods to multimodal models and (2) understanding the mechanistic differences between unimodal language models and crossmodal systems. By systematically reviewing current MMFM analysis techniques, we propose a structured taxonomy of interpretability methods, compare insights across unimodal and multimodal architectures, and highlight critical research gaps.

Problem

• Adapt LLM interpretability methods to multimodal models
• Understand mechanistic differences between unimodal and crossmodal systems
• Highlight research gaps in multimodal foundation models

Innovation

• Adapt LLM interpretability to MMFMs
• Compare unimodal and multimodal differences
• Taxonomy of MMFM interpretability methods