Prepared mind, fast response: A temporal decoupling framework for adaptive knowledge orchestration in open-domain dialogue

2025-10-09

AI Summary

In open-domain dialogue, balancing knowledge retrieval latency and response quality remains challenging: synchronous retrieval incurs high latency; lightweight models lack reasoning depth; and tool-augmented agents suffer from degraded interactivity due to blocking execution. This paper proposes a time-decoupled framework featuring a novel asynchronous knowledge coordination mechanism. A knowledge sufficiency evaluator dynamically assesses information needs, triggering parallel execution—lightweight front-end response generation and background asynchronous knowledge refinement—thereby eliminating synchronous blocking bottlenecks. The framework integrates real-time sufficiency assessment, progressive knowledge enhancement, and non-blocking tool invocation. On TopiOCQA, our approach reduces average response latency from 23.38 seconds to 1.09 seconds (↓95.3%) while achieving a GEval-C quality score of 0.613—comparable to heavy synchronous baselines (0.620) and significantly outperforming brute-force scaling approaches.

Technology Category

• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Natural Language Processing: Question Answering
• Search and Optimization: Learning to Search

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Agentic search
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

The latency-quality tradeoff is a fundamental constraint in open-domain dialogue AI systems, since comprehensive knowledge access necessitates prohibitive response delays. Contemporary approaches offer two inadequate solutions: lightweight instruct models achieve sub-second latency but lack reasoning depth, while tool-augmented ReAct agents enhance factuality through external knowledge at the cost of synchronous execution that blocks interaction during re- trieval processes. PMFR is thus proposed, with a tempo- ral decoupling framework that fundamentally resolves the contradiction through asynchronous knowledge orchestra- tion. PMFR employs three coordinated components: (1) a Knowledge Adequacy Evaluator for real-time sufficiency assessment, (2) a Lightweight Response Generator for imme- diate user interaction, and (3) an Asynchronous Knowledge Refinement Agent for background knowledge enhancement. This architecture maintains continuous conversational flow while progressively enriching knowledge coverage through intelligent triggering mechanisms. Evaluation results on Top- iOCQA demonstrate PMFR outperforms brute-force scaling: PMFR achieves 95.3% latency reduction (23.38s -> 1.09s) while preserving response quality comparable to heavyweight synchronous baselines (GEval-C: 0.613 vs. 0.620).

Problem

• Resolving latency-quality tradeoff in dialogue systems
• Enabling asynchronous knowledge access during conversations
• Maintaining conversational flow while enhancing knowledge coverage

Innovation

• Temporal decoupling framework for asynchronous knowledge orchestration
• Three coordinated components enable continuous conversational flow
• Lightweight response generator with background knowledge enhancement