Visual Decoding Operators: Towards a Compositional Theory of Visualization Perception

2026-04-02

AI Summary

Current research in visualization perception lacks a generalizable computational framework capable of predicting user performance across novel combinations of visualizations and tasks. This work proposes modeling visualization interpretation as a sequence of composable and reusable visual decoding operators. By decomposing tasks into chart-agnostic perceptual operations and characterizing the error properties of each operator through a hierarchical Bayesian model, the approach enables accurate prediction of performance on unseen tasks. In preregistered experiments involving PDF/CDF charts, the method successfully predicted both bias and variance in mean estimation from scatterplots using a specific operator composition strategy, significantly outperforming five alternative strategies. These results demonstrate the framework’s predictive validity and theoretical promise for generalizing across visualization types and perceptual tasks.

Abstract

Prior work on perceptual effectiveness has decomposed visualizations into smaller common units (e.g., channels such as angle, position, and length) to establish rankings. While useful, these decompositions lack the computational structure to predict performance for new visualization $\times$ task combinations, requiring new experiments for each. We propose an alternative unit of analysis: operationalizing quantitative visualization interpretation as sequences of composable visual decoding operators. Using probability density function (PDF) and cumulative distribution function (CDF) charts, we examine how chart-specific tasks can be decomposed into reusable, chart-agnostic perceptual operations and characterize their error profiles through hierarchical Bayesian modeling. We then test generalizability by composing learned operators to predict performance on a structurally different task: Moritz et al.'s [35] scatterplot mean-estimation experiment, where the chart type, chart dimensions, and analytic goal all differ from the learning conditions. With a pre-registered analysis plan, we compose operators under six candidate strategies and evaluate each against empirical data with no parameters fit to the response data. One strategy captures both bias and variance of observed responses; five alternatives fail in distinguishable ways. We argue that this decoding-operator-oriented approach to empirical visualization research and theory-building lays the groundwork for generative models that can predict a distribution of likely interpretations under different viewing conditions, new chart types, and new tasks. Free copy of this paper and supplemental materials: https://osf.io/prtfq; experiment interface: https://gleaming-dolphin-799fda.netlify.app/vis-decode-slider.

Problem

• visual decoding
• perceptual effectiveness
• visualization perception
• compositional theory
• generalizability

Innovation

• visual decoding operators
• compositional perception
• generalizable visualization theory
• hierarchical Bayesian modeling
• perceptual operators