Deep Model Merging: The Sister of Neural Network Interpretability -- A Survey

2024-10-16

AI Summary

This work investigates the intrinsic relationship between deep model merging and neural network interpretability, addressing the lack of a unified geometric foundation for understanding their interplay. Method: We systematically analyze empirical phenomena in model merging and identify four geometric properties of the loss landscape—mode convexity, determinacy, directionality, and connectivity—as shared structural determinants of merging efficacy and generalization robustness. We develop a unified framework integrating loss landscape analysis, neural representation modeling, and interpretability evaluation. Contribution/Results: Theoretically, we establish how landscape geometry governs the interpretability of internal representations and their stability across tasks. Our framework provides the first geometric interpretation paradigm for model merging and pioneers a novel interdisciplinary research direction at the intersection of “landscape geometry–interpretability–robustness,” offering principled insights into both model composition and trustworthy AI.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Machine Learning: Deep Neural Architectures and Foundation Models
• Computer Vision: Interpretability, Explainability, and Transparency

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We survey the model merging literature through the lens of loss landscape geometry to connect observations from empirical studies on model merging and loss landscape analysis to phenomena that govern neural network training and the emergence of their inner representations. We distill repeated empirical observations from the literature in these fields into descriptions of four major characteristics of loss landscape geometry: mode convexity, determinism, directedness, and connectivity. We argue that insights into the structure of learned representations from model merging have applications to model interpretability and robustness, subsequently we propose promising new research directions at the intersection of these fields.

Problem

• Analyzes loss landscape geometry in model merging
• Links merging observations to neural network training phenomena
• Explores applications in interpretability and robustness

Innovation

• Survey model merging via loss landscape geometry
• Identify four key loss landscape characteristics
• Link merging insights to interpretability and robustness