LLM Chemistry Estimation for Multi-LLM Recommendation

2025-10-04

AI Summary

Existing approaches to multi-LLM collaboration lack explicit modeling of inter-model complementarity or conflict, leading to unpredictable ensemble outcomes. Method: This paper introduces LLM Chemistry—a novel framework that formally defines “chemical reactions” among large language models, quantifies cooperative or adversarial interactions via dependency analysis, and establishes a task-aware theoretical model of synergistic effects. Contribution/Results: The framework enables predictive modeling and optimization of multi-model combinations across diverse tasks (e.g., classification, summarization, program repair), model scales, and problem complexities. Experiments demonstrate substantial improvements in ensemble performance; moreover, LLM Chemistry provides interpretable diagnostic tools and principled, dynamic model selection criteria for multi-LLM systems—thereby addressing a critical gap in the quantitative, mechanistic analysis of LLM collaboration.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Multi-LLM collaboration promises accurate, robust, and context-aware solutions, yet existing approaches rely on implicit selection and output assessment without analyzing whether collaborating models truly complement or conflict. We introduce LLM Chemistry -- a framework that measures when LLM combinations exhibit synergistic or antagonistic behaviors that shape collective performance beyond individual capabilities. We formalize the notion of chemistry among LLMs, propose algorithms that quantify it by analyzing interaction dependencies, and recommend optimal model ensembles accordingly. Our theoretical analysis shows that chemistry among collaborating LLMs is most evident under heterogeneous model profiles, with its outcome impact shaped by task type, group size, and complexity. Evaluation on classification, summarization, and program repair tasks provides initial evidence for these task-dependent effects, thereby reinforcing our theoretical results. This establishes LLM Chemistry as both a diagnostic factor in multi-LLM systems and a foundation for ensemble recommendation.

Problem

• Quantifying synergistic or antagonistic behaviors in LLM combinations
• Formalizing chemistry notion among LLMs through interaction dependencies
• Recommending optimal model ensembles based on measured chemistry

Innovation

• Quantifies synergy and antagonism in LLM combinations
• Recommends optimal ensembles based on interaction dependencies
• Analyzes chemistry effects across tasks and model profiles