Human-Computer Interaction and Visualization in Natural Language Generation Models: Applications, Challenges, and Opportunities

2024-10-11

AI Summary

Natural language generation (NLG) models suffer from poor interpretability due to architectural complexity and massive parameter counts, hindering their deployment in high-stakes decision-making. To address this, we propose the first unified human-computer interaction (HCI) and visualization classification framework specifically for NLG, systematically delineating three research paradigms and six core tasks that jointly address interpretability, controllability, and collaborative capability. By integrating principles from HCI design, eXplainable AI (XAI), information visualization, cognitive modeling, and NLG evaluation, we establish an interdisciplinary analytical framework that clarifies limitations of existing approaches. Furthermore, we introduce— for the first time in the large language model era—the “interactive–visualization co-enhancement” pathway. Our work provides both theoretical foundations and practical guidelines for developing trustworthy, controllable, and collaboratively capable NLG systems.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Humans and AI: Explainable AI (XAI) for Human Understanding
• Computer Vision: Language and Vision

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Natural language generation (NLG) models have emerged as a focal point of research within natural language processing (NLP), exhibiting remarkable performance in tasks such as text composition and dialogue generation. However, their intricate architectures and extensive model parameters pose significant challenges to interpretability, limiting their applicability in high-stakes decision-making scenarios. To address this issue, human-computer interaction (HCI) and visualization techniques offer promising avenues to enhance the transparency and usability of NLG models by making their decision-making processes more interpretable. In this paper, we provide a comprehensive investigation into the roles, limitations, and impact of HCI and visualization in facilitating human understanding and control over NLG systems. We introduce a taxonomy of interaction methods and visualization techniques, categorizing three major research domains and their corresponding six key tasks in the application of NLG models. Finally, we summarize the shortcomings in the existing work and investigate the key challenges and emerging opportunities in the era of large language models (LLMs).

Problem

• Enhancing NLG model transparency via HCI and visualization
• Addressing interpretability challenges in complex NLG architectures
• Exploring HCI and visualization roles in NLG usability

Innovation

• HCI enhances NLG model transparency
• Visualization improves NLG interpretability
• Taxonomy categorizes interaction and visualization methods