A Composable Agentic System for Automated Visual Data Reporting

2025-09-06

AI Summary

Traditional monolithic large language model (LLM)-based AI agents exhibit poor robustness, lack of auditability, and limited human intervention capability in automated visual data reporting. Method: This paper proposes a human-in-the-loop multi-agent hybrid architecture that decouples visualization design logic into a deterministic rule engine (Draco), delegates high-level reasoning and interactive guidance to an LLM, and integrates Observable for traceable, interactive report generation—simultaneously producing executable Marimo notebooks. Contribution/Results: Through modular design and strict responsibility separation, the system achieves a balance among automation, full auditability, and real-time expert intervention. The open-source implementation ensures full reproducibility and significantly enhances transparency, controllability, and efficiency in human–AI collaboration for data analysis.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-in-the-loop Machine Learning
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

To address the brittleness of monolithic AI agents, our prototype for automated visual data reporting explores a Human-AI Partnership model. Its hybrid, multi-agent architecture strategically externalizes logic from LLMs to deterministic modules, leveraging the rule-based system Draco for principled visualization design. The system delivers a dual-output: an interactive Observable report with Mosaic for reader exploration, and executable Marimo notebooks for deep, analyst-facing traceability. This granular architecture yields a fully automatic yet auditable and steerable system, charting a path toward a more synergistic partnership between human experts and AI. For reproducibility, our implementation and examples are available at https://peter-gy.github.io/VISxGenAI-2025/.

Problem

• Addresses brittleness of monolithic AI agents
• Explores Human-AI Partnership for visual reporting
• Delivers dual-output interactive and executable reports

Innovation

• Hybrid multi-agent architecture externalizes logic
• Leverages rule-based Draco for visualization design
• Dual-output system with interactive and executable components