A Review on Large Language Models for Visual Analytics

2025-03-19

AI Summary

This paper addresses critical challenges in the deep integration of large language models (LLMs) with visual analytics—namely, ambiguous task boundaries, fragmented technical approaches, and absent ethical safeguards. Methodologically, it establishes the first LLM-empowered visual analytics technology taxonomy and SWOT framework, proposing a classification system spanning 12 core tasks. It empirically benchmarks leading platforms (e.g., LIDA, Chat2VIS) and multimodal models (e.g., ChartLlama, CharXIV), while innovatively unifying natural language understanding, text-to-chart generation, and human-AI collaborative interaction—augmented by dual-dimensional constraints: ethical principles and methodological rigor. The study identifies three fundamental bottlenecks: computational overhead, representational bias, and data privacy risks. Its contributions include a theoretically grounded, practice-oriented foundation for developing trustworthy, interpretable, and human-centered visualization AI systems.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Language and Vision
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

This paper provides a comprehensive review of the integration of Large Language Models (LLMs) with visual analytics, addressing their foundational concepts, capabilities, and wide-ranging applications. It begins by outlining the theoretical underpinnings of visual analytics and the transformative potential of LLMs, specifically focusing on their roles in natural language understanding, natural language generation, dialogue systems, and text-to-media transformations. The review further investigates how the synergy between LLMs and visual analytics enhances data interpretation, visualization techniques, and interactive exploration capabilities. Key tools and platforms including LIDA, Chat2VIS, Julius AI, and Zoho Analytics, along with specialized multimodal models such as ChartLlama and CharXIV, are critically evaluated. The paper discusses their functionalities, strengths, and limitations in supporting data exploration, visualization enhancement, automated reporting, and insight extraction. The taxonomy of LLM tasks, ranging from natural language understanding (NLU), natural language generation (NLG), to dialogue systems and text-to-media transformations, is systematically explored. This review provides a SWOT analysis of integrating Large Language Models (LLMs) with visual analytics, highlighting strengths like accessibility and flexibility, weaknesses such as computational demands and biases, opportunities in multimodal integration and user collaboration, and threats including privacy concerns and skill degradation. It emphasizes addressing ethical considerations and methodological improvements for effective integration.

Problem

• Integration of LLMs with visual analytics for enhanced data interpretation.
• Evaluation of tools like LIDA and Chat2VIS for visualization and insight extraction.
• Addressing ethical and computational challenges in LLM-visual analytics synergy.

Innovation

• Integrates LLMs with visual analytics tools
• Enhances data interpretation and visualization techniques
• Evaluates multimodal models for automated reporting