DRAGON: Robust Classification for Very Large Collections of Software Repositories

2026-02-09

AI Summary

This work addresses the limitations of existing software repository classification methods, which rely heavily on metadata such as README files and thus suffer from poor applicability in large-scale real-world scenarios due to frequent information gaps. To overcome this, we propose DRAGON, a novel approach that leverages only lightweight signals from version control systems—such as file and directory names—with optional integration of README content when available. DRAGON demonstrates remarkable robustness, with performance degrading by merely 6% in the absence of README files, and its predictions often yield semantically related “near-misses” that retain practical retrieval value. Evaluated on the largest open-source repository classification dataset to date, comprising 825,000 repositories, DRAGON achieves an F1@5 score of 60.8%, outperforming the current state-of-the-art by 6 percentage points and confirming its effectiveness and stability.

Technology Category

• Computer Vision: Image and Video Retrieval
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Search and Optimization: Metareasoning and Metaheuristics

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

The ability to automatically classify source code repositories with''topics''that reflect their content and purpose is very useful, especially when navigating or searching through large software collections. However, existing approaches often rely heavily on README files and other metadata, which are frequently missing, limiting their applicability in real-world large-scale settings. We present DRAGON, a repository classifier designed for very large and diverse software collections. It operates entirely on lightweight signals commonly stored in version control systems: file and directory names, and optionally the README when available. In repository classification at scale, DRAGON improves F1@5 from 54.8% to 60.8%, surpassing the state of the art. DRAGON remains effective even when README files are absent, with performance degrading by only 6% w.r.t. when they are present. This robustness makes it practical for real-world settings where documentation is sparse or inconsistent. Furthermore, many of the remaining classification errors are near misses, where predicted labels are semantically close to the correct topics. This property increases the practical value of the predictions in real-world software collections, where suggesting a few related topics can still guide search and discovery. As a byproduct of developing DRAGON, we also release the largest open dataset to date for repository classification, consisting of 825 thousand repositories with associated ground-truth topics, sourced from the Software Heritage archive, providing a foundation for future large-scale and language-agnostic research on software repository understanding.

Problem

• repository classification
• software repositories
• topic labeling
• metadata scarcity
• large-scale software collections

Innovation

• repository classification
• robustness
• lightweight signals
• large-scale software analysis
• language-agnostic