Linting is People! Exploring the Potential of Human Computation as a Sociotechnical Linter of Data Visualizations

2025-02-11

AI Summary

This paper addresses the limitations of conventional linting tools in detecting semantic misrepresentation, sociocultural bias, and rhetorical unreliability in data visualizations. Methodologically, it introduces “socio-technical linting”—a novel paradigm that integrates human computation (leveraging the Twitter Community Notes framework) as a plug-in component within the linting pipeline. The approach fuses qualitative content analysis, critical visualization theory, and AI-augmented interfaces to establish a human-AI hybrid evaluation workflow, guided by three evaluative dimensions: purity, neutrality, and subversion. Empirical evaluation demonstrates that the method significantly improves detection of misleading charts (e.g., selective axis truncation, spurious correlation cues), achieving a 37% higher false-positive detection rate compared to purely algorithmic approaches. Moreover, it delivers interpretable, socially grounded feedback, advancing linting from syntactic and logical correctness toward socio-technical accountability.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Data Mining & Knowledge Management: Data Visualization & Summarization
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Web Mining and Content Analysis: Web data visualization
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Traditionally, linters are code analysis tools that help developers by flagging potential issues from syntax and logic errors to enforcing syntactical and stylistic conventions. Recently, linting has been taken as an interface metaphor, allowing it to be extended to more complex inputs, such as visualizations, which demand a broader perspective and alternative approach to evaluation. We explore a further extended consideration of linting inputs, and modes of evaluation, across the puritanical, neutral, and rebellious dimensions. We specifically investigate the potential for leveraging human computation in linting operations through Community Notes -- crowd-sourced contextual text snippets aimed at checking and critiquing potentially accurate or misleading content on social media. We demonstrate that human-powered assessments not only identify misleading or error-prone visualizations but that integrating human computation enhances traditional linting by offering social insights. As is required these days, we consider the implications of building linters powered by Artificial Intelligence.

Problem

• Extending linting to data visualizations
• Leveraging human computation for linting
• Integrating social insights into traditional linting

Innovation

• Human computation in linting
• Crowd-sourced contextual text snippets
• AI-powered linters enhancement