Replication Studies: Not Just a Copy

2026-09-16

AI Summary

本文提出REPVIS2设计空间，通过八个维度描述复制研究与参考研究的关系，以解决复制研究设计难以描述和比较的问题。

Abstract

Replication studies revisit previous experimental findings for multiple purposes, including assessing reliability, exploring generalisability, and evaluating research methods. Such studies entail design decisions that a single replication label cannot fully capture, making their designs difficult to describe and compare. We present REPVIS2, a validated design space that describes how a replication study relates to a reference study across eight practical dimensions, each coded as identical, similar, or different. We refined our initial design space, REPVIS1, by applying it to a corpus of replication studies and validated REPVIS2 against a separate corpus. We then characterised 86 replication studies from 51 papers in visualisation through paired reading of the replication and reference reports. Studies often retained the task while changing the procedure, interface, environment, participant population, evidence, or analysis. Additions beyond the replication core were also common. REPVIS2 makes replication design explicit for characterisation, reporting, and planning. An interactive visualisation and supplementary materials are available at https://replication-study.github.io/.

Problem

• replication studies
• design space
• comparison
• description

Innovation

• Replication Studies
• Design Space
• REPVIS2
• Validation