Reproducibility Beyond Artifacts: Interactional Support for Collaborative Machine Learning

2026-04-07

AI Summary

This work addresses reproducibility challenges in collaborative machine learning, which often stem not merely from missing artifacts but from team misalignments in interpreting prior work, inconsistent component evolution, and difficulties in reconstructing experimental intent. To tackle these issues, the authors propose a novel two-layer socio-technical architecture that integrates interactive support into reproducibility frameworks for the first time. The lower layer leverages a data-centric ML management system with full lifecycle provenance tracking, while the upper layer employs an AI-mediated semantic interface to enable structured collaboration, explanatory discourse, and consensus building. Deployed over 19 months in a clinical research setting, the system effectively identified and mitigated persistent interaction breakdowns, substantially enhancing shared team understanding and experimental reproducibility.

Abstract

Machine learning (ML) reproducibility is often framed as a problem of incomplete artifact recording. This framing leads to systems that prioritize capturing datasets, code, configurations, and execution environments.However, in collaborative and interdisciplinary ML projects, reproducibility failures often arise not only from missing artifacts but from difficulties in interpreting prior work, aligning evolving components, and reconstructing experimental intent over time. Drawing on a 19-month deployment of a data-centric ML management system in a clinical research project, we identify recurring interactional breakdowns that persist despite comprehensive structural traceability. Based on these findings, we propose a two-layer socio-technical ML management system combining lifecycle-aware artifact infrastructure with an interactional layer designed to mediate coordination, explanation, and shared understanding. We discuss how an AI-mediated semantic interface reframes reproducibility as an ongoing socio-technical accomplishment rather than a static property of recorded traces, and outline implications for human-centered ML infrastructure design.

Problem

• reproducibility
• collaborative machine learning
• interactional breakdowns
• experimental intent
• socio-technical systems

Innovation

• reproducibility
• socio-technical system
• interactional layer
• AI-mediated interface
• collaborative machine learning