Marks, Channels, and Dead Ends: Stop Running Graphical Perception Studies and Start Modeling Visualizations as Images

2026-08-07

AI Summary

Traditional research on graphical perception has predominantly evaluated visualizations from an encoding perspective, often overlooking the fact that the human visual system processes pixel-based images, thereby creating a disconnect between evaluation and actual perception. This work proposes treating visualizations as images and, for the first time, systematically integrates summary statistic vision theory by employing computational vision models that take pixels as input to model the perceptual process from the decoding end. The approach not only successfully reproduces established findings in graphical perception but also sensitively predicts perceptual changes induced by subtle variations in data distributions or design choices, demonstrating the effectiveness and potential of image-based vision models for evaluating visualizations.

Abstract

Graphical perception studies are the visualization community's preferred tool for evaluating visualizations. By measuring how accurately people interpret arrangements of visual marks and channels, they aim to establish best practices for visual encoding. We argue that this model is fundamentally flawed, and no amount of additional empirical studies will fix it. Visualization theory frames effectiveness at the level of the encoder: which data-to-visual mappings work best in a given context. Human perception, however, operates as a fundamentally different decoder at the level of retinal images. This encoder-decoder asymmetry means that experimental results and guidelines can be poor predictors of perceptual performance. Moreover, the image reaching the visual system emerges from interactions among encoding rules, input data, and micro-design parameters--factors largely invisible to encoding theory. Consequently, small changes in data distributions or design variations can substantially alter perception even when the nominal encoding remains unchanged. We argue that visualizations should instead be studied as images and evaluated using computational models of human vision that take pixels as input. Such models capture the perceptual representations the visual system actually constructs, shifting evaluation toward the decoder rather than abstract encoding specifications. This approach is scalable, human-grounded, and sensitive to emergent image properties that both encoding theory and graphical perception studies miss. We first describe weaknesses of the current paradigm and propose a theory of visualization perception grounded in summary-statistical accounts of vision. We then show how image-based vision models can predict visualization discriminability in scatterplots while reproducing established results. We close by outlining a research agenda for vision-based visualization evaluation.

Problem

• graphical perception
• visualization evaluation
• visual encoding
• human vision
• image-based modeling

Innovation

• graphical perception
• visualization evaluation
• computational vision models
• encoder-decoder asymmetry
• image-based modeling