RepoTrace: Browser-Assisted Evidence Collection for GitHub Research Datasets

2026-07-06

AI Summary

This work addresses the challenges of manual GitHub dataset construction, where evidence and annotations are scattered across multiple sources, hindering traceability and collaborative auditing. The authors propose the first end-to-end framework that integrates browser-assisted data collection, local structured storage, and decision provenance tracking. Leveraging a Chrome extension to capture page snapshots, comments, and labels, the system couples an Express backend with a React dashboard to consolidate all research decisions into a unified SQLite workspace, tightly linking evidence with explanatory context. Evaluated on the Matplotlib project, the framework successfully collected 22 snapshots, 38 comments, and 98 annotations across 20 issues, fully preserving the research process and enabling conflict simulation. This approach substantially enhances the auditability and reproducibility of dataset construction.

Abstract

Empirical software engineering studies frequently build datasets from GitHub issues and pull requests. In many projects, researchers inspect pages in a browser, copy selected fields into spreadsheets, keep side notes in separate documents, and later run scripts to normalize or export the data. This workflow is flexible, but the page evidence, the research codes, and the rationale behind each decision end up spread across tabs and files, which leaves provenance, update tracking, and multi-reviewer labeling hard to audit. RepoTrace is a browser-assisted research tool that collects GitHub issue and pull-request evidence into a local SQLite-backed workspace. It combines a Chrome side-panel extension, an Express backend, and a React dashboard to capture page snapshots, comments, labels, notes, screening and labeling decisions, refresh history, and scoped exports, keeping the source evidence and the research interpretation linked together. A validation pass collected and checked 20 Matplotlib issues across two study projects. The resulting dataset preserves 22 snapshots, 38 comments, 20 research notes, 98 annotations, 20 screening reviews, 20 fix-evidence entries, and 4 simulated unresolved consensus conflicts. The results show that RepoTrace can support a complete local evidence-collection workflow for manually constructed GitHub issue and pull-request datasets.

Problem

• GitHub dataset
• evidence collection
• provenance tracking
• manual annotation
• research reproducibility

Innovation

• browser-assisted research
• evidence collection
• GitHub dataset
• provenance tracking
• manual annotation