Exploring Information Seeking Agent Consolidation

2026-01-31

AI Summary

Existing information-seeking agents are typically confined to a single source of information, limiting their ability to generalize across domains and scale effectively. This work proposes unifying heterogeneous agents into a single foundation model and systematically compares two integration strategies: data-level joint training and parameter-level fusion. Through techniques such as multi-source data mixing, modeling task heterogeneity, and consensus mechanisms, the study reveals that fine-grained merging and task-awareness are critical to the success of parameter-level fusion. Experimental results demonstrate that data-level integration provides a robust and stable baseline, while parameter-level methods, though computationally efficient, are susceptible to behavioral interference and highly dependent on the identified key design factors for optimal performance.

Technology Category

• Multiagent Systems: Other Foundations of Multi Agent Systems
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Information-seeking agents have emerged as a powerful paradigm for solving knowledge-intensive tasks. Existing information-seeking agents are typically specialized for open web, documents, or local knowledge bases, which constrains scalability and cross-domain generalization. In this work, we investigate how to consolidate heterogeneous information-seeking agents into a single foundation agentic model. We study two complementary consolidation strategies: data-level consolidation, which jointly trains a unified model on a mixture of domain-specific datasets, and parameter-level consolidation, which merges independently trained agent models at the parameter level. Our analysis compares these approaches in terms of performance retention, cross-domain generalization, and interference across information-seeking behaviors. Our results show that data-level consolidation remains a strong and stable baseline, while parameter-level consolidation offers a promising, efficient alternative but suffers from interference and robustness challenges. We further identify key design factors for effective agent consolidation at the parameter level, including fine-grained merging granularity, awareness of task heterogeneity, and principled consensus strategy.

Problem

• information-seeking agents
• agent consolidation
• cross-domain generalization
• heterogeneous agents
• foundation agentic model

Innovation

• agent consolidation
• information-seeking agents
• parameter merging
• cross-domain generalization
• foundation agentic model