Deforking the World of Code: A Project-Provenance Map that Recovers Cross-Forge Fork Families that Platform Graphs Cannot See

2026-06-28

AI Summary

This study addresses the challenge that existing code hosting platforms fail to identify cross-platform forking relationships, leading to severe inflation of project popularity metrics due to duplicate counting. To resolve this, the authors propose a de-forking method grounded in globally shared commit relationships, employing star encoding, parallel Louvain clustering, and a cluster-size truncation strategy to systematically reconstruct cross-platform fork families. Their approach enables, for the first time, the identification of root projects outside GitHub, revealing that 5.41% of multi-platform fork families and 1.51% of forks originate from non-GitHub roots. The released de-forking mapping, p2PFull, covers World of Code V2604, achieves 99.01% edge consistency with GitHub’s native fork graph, and includes an exclusion list of 134 million child repositories alongside 455,000 hard-split records.

Abstract

Forks share git history, so a commit surfaces in many repositories and any spread- or popularity-based measure over raw repositories is inflated by orders of magnitude. We release a curated deforking map for the World of Code (WoC) version V2604: p2PFull, which collapses every raw repository p into the deforked project P to which it belongs, built from the global shared-commit relation (51.79M shared-commit groups) via a hub-node star encoding and parallel Louvain clustering, plus capped variants (cap250/cap500) that bound mega-cluster size. The naive shared-history union over-merges: the project graph welds unrelated software into giant clusters (largest uncapped cluster 861,948 repositories, bridged by shared-commit groups as large as 267,200), for the same structural reason author-identity graphs do. A cheap size cap removes the boilerplate-hub bridges; a structural-bridge diagnostic, the cut that dissolved the analogous author mega-cluster, run here but deliberately not applied, shows the post-cap residual is genuine vendored history, robust to the cut, so we leave it intact. We validate the map against GitHub's declared fork graph reconstructed from GHArchive ForkEvents, finding 99.01% edge agreement conditional on both repositories being in WoC. Disagreements fall into two classes: a completeness byproduct (edges GitHub asserts but WoC has not ingested) and the central contribution, WoC-only fork families that GitHub's platform graph cannot represent, including 5.41% multi-forge families and 1.51% whose fork root is not on GitHub. We additionally release a refreshed fork-exclusion list (134.1M children, 3.4x the GHTorrent-era 39.5M) and a detached-fork inventory (455,550 hard-detached edges; 240,441 genuine independent origins). All artifacts are a self-contained, independently hosted replication package keyed to the WoC V2604 collection.

Problem

• fork detection
• cross-forge
• project provenance
• code repositories
• shared commit history

Innovation

• deforking
• cross-forge forks
• shared-commit clustering
• project provenance
• code ecosystem analysis