Chasing Meaning and/or Insight? A Survey on Evaluation Practices at the Intersection of Visualization and the Humanities

2026-01-28

AI Summary

This study addresses the persistent evaluative tension in humanities visualization research between analytical “insight” and interpretive “meaning,” a challenge inadequately met by existing assessment frameworks rooted in non-empiricist epistemologies. Through a systematic literature review of evaluation practices across 171 design studies, the work reveals a prevalent overreliance on singular methodological approaches that constrains rigor and relevance. In response, it proposes a pluralistic triangulation paradigm that integrates conventional validation techniques with the hermeneutic depth characteristic of humanities scholarship. This integrative approach substantially enhances both the methodological robustness and disciplinary appropriateness of evaluations. Building on this foundation, the study formulates a new set of evaluation criteria tailored specifically to humanities visualization, thereby advancing the reconceptualization of assessment standards in this interdisciplinary domain.

Technology Category

• Application Domains: Humanities & Computational Social Science
• Search and Optimization: Evaluation and Analysis
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data visualization
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success

Abstract

The intersection of visualization and the humanities (VIS*H) is marked by a tension between chasing analytical"insight"and interpretive"meaning."The effectiveness of visualization techniques hinges on established evaluation frameworks that assess both analytical utility and communicative efficacy, creating a potential mismatch with the non-positivist, interpretive aims of humanities scholarship. To examine how this tension manifests in practice, we systematically surveyed 171 VIS*H design studies to analyze their evaluation workflows and rigor according to standard practice. Our findings reveal recurring flaws, such as an over-reliance on monomethod approaches, and show that higher-quality evaluations emerge from workflows that effectively triangulate diverse evidence. From these findings, we derive recommendations to refine quality and validation criteria for humanities visualizations, and juxtapose them to ongoing critical debates in the field, ultimately arguing for a paradigm shift that can reconcile the advantages of established validation techniques with the interpretive depth required for humanistic inquiry.

Problem

• visualization
• humanities
• evaluation
• insight
• meaning

Innovation

• visualization evaluation
• humanities
• triangulation
• interpretive meaning
• design study