Decoupling Search from Reasoning: A Vendor-Agnostic Grounding Architecture for LLM Agents

2026-06-17

AI Summary

This work addresses the challenges in real-time search for large language model (LLM) agents, where tightly coupled retrieval strategies, provider dependencies, evidence injection mechanisms, and cost-latency trade-offs hinder debuggability, reusability, and output efficiency. To resolve this, the authors propose Decoupled Search Grounding (DSG), an architecture that reconceptualizes search grounding as an optimizable interface boundary rather than an intrinsic model capability. DSG introduces an MCP-compatible gateway to deliver a vendor-agnostic, configurable grounding layer, enabling source-aware context rendering, hybrid caching, tunable retrieval depth, and fallback mechanisms. Experiments demonstrate that on SimpleQA, DSG achieves 86.1% accuracy—approaching the native 87.7%—with 91% lower search cost, 99.4% cache hit rate, and 68% reduced latency; in e-commerce query tasks, it reduces search costs by over 98% while matching or slightly exceeding baseline accuracy.

Abstract

Production LLM agents increasingly depend on real-time search, yet native search grounding bundles retrieval policy, provider choice, evidence injection, cost, latency, and generation behavior behind a single model-provider boundary. This coupling makes grounding hard to inspect, tune, reuse, or port, and can trigger Search-Induced Verbosity that breaks strict output contracts. We present Decoupled Search Grounding (DSG), a vendor-agnostic boundary that moves grounding outside the reasoning model through an MCP-compatible gateway, exposing provider routing, source-aware context rendering, configured fallback, retrieval-depth control, and exact plus semantic caching as first-class controls. Across five frontier models on SimpleQA, FreshQA, and HotpotQA, native search leads on recency-sensitive FreshQA, but DSG exposes a stronger frontier when control matters: on SimpleQA it nearly matches native accuracy (86.1% vs. 87.7%) at 91% lower search cost, preserves concise answer contracts, and reaches a 99.4% warm-cache hit rate with 68% lower latency. Deployed as a shared production grounding layer for large-scale agentic workloads with interchangeable models, DSG matches or slightly exceeds native-search accuracy on an e-commerce query-understanding (QIU) workload while cutting search cost by over 98%. Real-time grounding is best treated as an optimizable interface boundary, not a fixed model feature.

Problem

• search grounding
• LLM agents
• vendor coupling
• real-time retrieval
• output verbosity

Innovation

• Decoupled Search Grounding
• vendor-agnostic
• search grounding
• MCP-compatible gateway
• semantic caching