🤖 AI Summary
This work addresses the state desynchronization between natural language queries and dashboard interactions in multi-step business intelligence (BI) analysis by proposing the first agent-based digital twin framework. The approach couples large language model (LLM) agents with executable dashboard states and reconstructs a shared analytical context through unified interaction logs, thereby ensuring consistency across dialogue, user actions, semantic alignment, and provenance tracking. Innovatively leveraging a digital twin mechanism, the framework enables state-aware analytical summarization and traceable context management. Experimental results demonstrate a 20.0% absolute improvement in exact-match accuracy (from 43.3% to 63.3%), a partial-match accuracy of 70.8%, and a reduced timeout rate of 10.0%. User studies further confirm high task accuracy and a positive interactive experience.
📝 Abstract
Business intelligence (BI) increasingly combines dashboard interaction with LLM-based assistance, but these two modes often fall out of sync during multi-step analysis. As users switch between direct dashboard manipulation and natural-language queries, it becomes difficult to preserve a consistent analytical state across filters, hierarchies, metrics, and chart context. We present TwinBI, an agentic digital-twin framework that couples an LLM-based agent system with an executable BI dashboard state. TwinBI unifies conversational interaction, dashboard manipulation, semantic grounding, and provenance tracking through a shared analytical state reconstructed from a unified interaction log. It also exposes artifacts such as schema views, SQL, logs, and an /insights command for state-grounded analytical summaries. We evaluate TwinBI in two complementary ways. In a controlled A/B benchmark with the same backbone agent, TwinBI improves exact-match accuracy from 43.3% to 63.3%, partial-credit accuracy from 48.3% to 70.8%, and substantially reduces timeout rate from 40.0% to 10.0% relative to Dashboard alone. In a usability study, participants benefited from the integrated dashboard-and-chat workflow, with high task accuracy, moderate workload, and favorable ratings for state-aware interaction mechanisms. These results suggest that TwinBI improves both agent-level analytical reliability and user-facing analytical support by turning visible dashboard state into richer actionable context. Our dataset and source code are available at: https://github.com/simonjisu/TwinBI