Understanding How Humans Inject Knowledge into Machine Learning Workflows through Visual Analytics

2026-07-01

AI Summary

This study investigates the mechanisms and pathways through which human domain knowledge is integrated into machine learning (ML) workflows via visual analytics. Building upon a systematic review of over 200 VIS4ML papers, the authors develop a coding framework encompassing four dimensions: machine learning, visualization, interaction, and action. By synthesizing perspectives from model construction and information-theoretic cost–benefit analysis, they propose the first unified explanatory framework for knowledge injection in ML. The work elucidates the pivotal role of interactive visualization in optimizing ML workflows and systematically maps the multidimensional pathways through which human expertise is incorporated. This contribution provides both theoretical grounding and empirical foundations for advancing research and practice in the VIS4ML community.

Abstract

Visual analytics (VA) plays an increasingly important role in supporting machine learning (ML) workflows. In the field of visualization, such approaches and techniques are referred to as VIS4ML. While ML models are mostly learned automatically, the corresponding ML workflows receive a variety of human inputs, such as data labelling, feature engineering, model architecture designing, hyper-parameter tuning, and so on. In this work, we surveyed over 200 VIS4ML papers to gain an understanding of how humans inject their knowledge into ML workflows through interactive visualization. We collected a corpus of VIS4ML papers from the IEEE VIS conferences in the past decade. We developed a coding scheme to facilitate the literature research from four perspectives: characteristics of ML, visualization, interaction, and actions. The analysis of the coded dataset allows us to observe different pathways that transfer human knowledge to ML workflows via interactive visualization. Building on the analysis, we explain the phenomena of VIS4ML using the conceptual model that views VA as model building and the information-theoretic cost-benefit analysis that reasons VA as for optimizing ML workflows. This work provides unequivocal evidence showing the merits of using VA in ML workflows. The full list of surveyed papers, along with all analysis results and figures, is available at https://vis4ml4hd.github.io/ml-knowledge-inject-va/.

Problem

• visual analytics
• machine learning workflows
• human knowledge injection
• VIS4ML
• interactive visualization

Innovation

• visual analytics
• machine learning workflows
• human-in-the-loop
• knowledge injection
• interactive visualization