nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow

2025-02-07

AI Summary

Current large language models (LLMs) exhibit weak reasoning capabilities and poor code robustness in generating visualizations from multi-table data. To address this, we propose an end-to-end natural language-to-visualization generation framework. Our method introduces a novel three-agent collaborative architecture—comprising a Processor, a Composer, and a Verifier—integrated with database context-aware filtering, visualization plan generation, and Python code execution-based validation. This design significantly enhances comprehension and generation of complex cross-table queries. On the VisEval benchmark, our approach achieves absolute accuracy improvements of 7.88% (single-table) and 9.23% (multi-table) over prior methods; qualitative analysis further indicates a consistent ~20% improvement in generated visualization quality. This work establishes a scalable, verifiable paradigm for semantic-driven visualization over heterogeneous, multi-source structured data.

Technology Category

• Computer Vision: Visual Reasoning & Symbolic Representations
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Natural Language to Visualization (NL2Vis) seeks to convert natural-language descriptions into visual representations of given tables, empowering users to derive insights from large-scale data. Recent advancements in Large Language Models (LLMs) show promise in automating code generation to transform tabular data into accessible visualizations. However, they often struggle with complex queries that require reasoning across multiple tables. To address this limitation, we propose a collaborative agent workflow, termed nvAgent, for NL2Vis. Specifically, nvAgent comprises three agents: a processor agent for database processing and context filtering, a composer agent for planning visualization generation, and a validator agent for code translation and output verification. Comprehensive evaluations on the new VisEval benchmark demonstrate that nvAgent consistently surpasses state-of-the-art baselines, achieving a 7.88% improvement in single-table and a 9.23% improvement in multi-table scenarios. Qualitative analyses further highlight that nvAgent maintains nearly a 20% performance margin over previous models, underscoring its capacity to produce high-quality visual representations from complex, heterogeneous data sources.

Problem

• Automate NL2Vis with agent workflow
• Handle complex multi-table queries
• Improve visualization generation accuracy

Innovation

• Collaborative agent workflow for NL2Vis
• Processor, composer, validator agents
• Improves visualization accuracy significantly