State-Grounded Multi-Agent Synthetic Data Generation for Tool-Augmented LLMs

2026-06-15

AI Summary

This work addresses the scarcity of large-scale, multi-turn, tool-grounded dialogue data for tool-augmented large language model (LLM) agents, which is hindered by high annotation costs, privacy constraints, and limited real-world availability. To overcome this, the authors propose StateGen, a synthetic dialogue generation platform featuring a four-role LLM collaborative architecture comprising a user simulator, the agent under test, a state-grounded tool simulator, and a multi-dimensional evaluator. Its key innovations include an authoritative state manager that maintains a structured world state and enforces a “backend-as-truth” principle to eliminate tool-calling hallucinations, alongside a 23-dimensional personality vector to drive conversational diversity and support hierarchical multi-agent extension. Evaluated on 64,698 dialogues, StateGen achieves a tool-calling hallucination score of 9.66/10—significantly outperforming eight existing systems—and demonstrates no memory bias.

Abstract

Training tool-augmented LLM agents requires large corpora of multi-turn, tool-grounded conversational data that is expensive to annotate, privacy-constrained in production settings, and largely absent from public datasets. We present StateGen, a synthetic data generation platform that produces scored, reasoning-trace-rich training conversations by orchestrating a four-role LLM loop: a persona-conditioned user simulator, an agent under test, a state-grounded tool simulator, and a multi-axis LLM judge. The key architectural contribution is an authoritative state manager that maintains a structured world-state object across turns, enforcing a backend-is-truth invariant that eliminates the dominant class of tool-call hallucinations by construction. StateGen extends naturally to hierarchical multi-agent settings by declaring sub-agents as tools, all sharing a single state object. We report results on 64,698 evaluated conversations across three production corpora: tool-call hallucination scores reach 9.66/10, the system supports persona-driven variation via a 23-dimensional trait vector, and a cleanly separated train and golden evaluation set split confirms the data is not memorization bait (per-criterion gap analysis). Comparison with eight external systems shows that no single publicly available platform combines multi-turn generation, state-grounded tool simulation, hierarchical multi-agent support, and built-in judge scoring.

Problem

• tool-augmented LLMs
• multi-agent
• synthetic data generation
• tool-grounded conversations
• data scarcity

Innovation

• state-grounded
• synthetic data generation
• tool-augmented LLMs
• multi-agent simulation
• hallucination mitigation