Behavior Modeling for Training-free Building of Private Domain Multi Agent System

2025-11-13

AI Summary

To address challenges in deploying private-domain multi-agent systems—including tool heterogeneity, domain-specific terminology, restricted APIs, and complex governance—this paper proposes a fine-tuning-free, documentation-driven multi-agent dialogue framework. Methodologically, it integrates tools automatically and interprets domain jargon via structured tool specifications, domain-aware prompting, API documentation retrieval augmentation, and behavioral modeling; a collaborative architecture comprising dialogue agents, tool invocation agents, and orchestrators enables dynamic adaptation to private APIs and governance constraints. Its key contribution lies in eliminating synthetic data training: instead, it auto-generates evaluation datasets and dialogue policies directly from documentation, ensuring cross-domain robustness and sustainable evolution. Experiments demonstrate that, without retraining, the framework significantly improves tool-call accuracy and dialogue quality in private settings, effectively mitigating domain drift.

Technology Category

• Multiagent Systems: Adversarial Agents
• Natural Language Processing: Conversational AI/Dialog Systems
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The rise of agentic systems that combine orchestration, tool use, and conversational capabilities, has been more visible by the recent advent of large language models (LLMs). While open-domain frameworks exist, applying them in private domains remains difficult due to heterogeneous tool formats, domain-specific jargon, restricted accessibility of APIs, and complex governance. Conventional solutions, such as fine-tuning on synthetic dialogue data, are burdensome and brittle under domain shifts, and risk degrading general performance. In this light, we introduce a framework for private-domain multi-agent conversational systems that avoids training and data generation by adopting behavior modeling and documentation. Our design simply assumes an orchestrator, a tool-calling agent, and a general chat agent, with tool integration defined through structured specifications and domain-informed instructions. This approach enables scalable adaptation to private tools and evolving contexts without continual retraining. The framework supports practical use cases, including lightweight deployment of multi-agent systems, leveraging API specifications as retrieval resources, and generating synthetic dialogue for evaluation -- providing a sustainable method for aligning agent behavior with domain expertise in private conversational ecosystems.

Problem

• Building private domain multi-agent systems without training
• Overcoming heterogeneous tools and domain-specific jargon challenges
• Avoiding performance degradation from conventional fine-tuning methods

Innovation

• Adopts behavior modeling to avoid training requirements
• Integrates tools through structured specifications and instructions
• Enables scalable adaptation without continual retraining cycles