Accessible Text Descriptions for UpSet Plots

2025-03-21

AI Summary

This work addresses the challenge faced by blind and low-vision (BLV) users in interpreting complex set visualizations—specifically UpSet plots. We propose the first automated text-description generation method tailored to UpSet plots. Unlike naive approaches that directly prompt large language models (LLMs), our method systematically analyzes real-world UpSet plot structural patterns and designs a domain-specific, interpretable, rule-based description framework. Evaluated via semi-structured interviews and controlled experiments with 11 BLV participants, the generated descriptions were confirmed to be informationally sufficient. Sighted users relying solely on these textual descriptions achieved significantly higher task accuracy and efficiency in UpSet plot analysis compared to zero-shot LLM outputs. This work fills a critical gap in accessible UpSet visualization research and establishes a reproducible, scalable, rule-driven paradigm for scientific chart accessibility.

Technology Category

• Computer Vision: Language and Vision
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Humans and AI: AI for Accessibility

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Web Mining and Content Analysis: Web data visualization
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Data visualizations are typically not accessible to blind and low-vision (BLV) users. Automatically generating text descriptions offers an enticing mechanism for democratizing access to the information held in complex scientific charts, yet appropriate procedures for generating those texts remain elusive. Pursuing this issue, we study a single complex chart form: UpSet plots. UpSet Plots are a common way to analyze set data, an area largely unexplored by prior accessibility literature. By analyzing the patterns present in real-world examples, we develop a system for automatically captioning any UpSet plot. We evaluated the utility of our captions via semi-structured interviews with (N=11) BLV users and found that BLV users find them informative. In extensions, we find that sighted users can use our texts similarly to UpSet plots and that they are better than naive LLM usage.

Problem

• Generate accessible text descriptions for UpSet plots
• Address lack of accessibility for BLV users in data visualizations
• Develop automated captioning system for complex scientific charts

Innovation

• Automatically generate text for UpSet plots
• Develop system via real-world pattern analysis
• Evaluate captions with BLV user interviews