Reflections on Traceability for Visualization Research

2026-04-15

AI Summary

Design-oriented visualization research often struggles to meet conventional reproducibility standards due to its inherent subjectivity, contextual dependence, and iterative nature, thereby limiting its transparency and rigor. To address this challenge, this work proposes “traceability” as a viable alternative to traditional reproducibility. It presents the first systematic theoretical framework centered on three core components—recording, reporting, and reading—and introduces tRRRacer, a supporting tool implementing this framework. Through collaborative autoethnography, the authors reflect on practical applications of traceability in design-oriented research, demonstrating its feasibility and yielding actionable principles alongside theoretical insights. This approach offers a novel pathway to enhance the rigor and transparency of such studies without relying on strict reproducibility criteria.

Technology Category

• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Computer Vision: Interpretability, Explainability, and Transparency
• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Algorithmic accountability and transparency on the web
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Decades of advocacy for reproducibility and replication have advanced open, transparent practices in the sciences. However, traditional notions of reproducibility fit poorly with design-oriented visualization research, where insights emerge through subjective, situated, and iterative work. So how can we ensure rigor and transparency in processes that are inherently unreproducible? To introduce transparency in design-oriented research, we propose to focus on traceability: surfacing the origin and development of research contributions based on rich sets of artifacts documenting the design process. We investigated traceability through a collaborative autoethnographic reflection that builds on several years of work exploring ways to make design-oriented research transparent. This exploration includes an experiment to build a tool to support traceability, which we called tRRRacer. The tRRRacer tool provided a testbed for us to operationalize the three tenets of a traceable process: (1) Record abundant, annotated artifacts representative of research activities; (2) Report curated research threads that articulate rationale and evolution of the process, allowing others to (3) Read via interfaces that help retrace claims and assess plausibility. Reflecting on our experiences, we contribute a theorization of traceability and reflections on how we might support it.

Problem

• traceability
• visualization research
• design-oriented research
• transparency
• reproducibility

Innovation

• traceability
• design-oriented research
• visualization
• tRRRacer
• research transparency