RePro: Leveraging Large Language Models for Semi-Automated Reproduction of Networking Research Results

2025-09-25

AI Summary

Reproducibility in networking research is severely hindered by scarce open-source implementations and the high manual effort required for paper replication. Method: This paper introduces RePro, the first semi-automated framework enabling cross-subdomain replication across diverse networking areas (e.g., SDN, network measurement, congestion control). RePro employs a three-stage pipeline—systematic parsing, code generation, and optimization verification—and integrates few-shot context learning with structured and semantic chain-of-thought (SCoT/SeCoT) prompting to enhance large language models’ (LLMs) generalization and accuracy in system-level code synthesis. Contribution/Results: Evaluated on five state-of-the-art LLMs and multiple networking domains, RePro significantly reduces replication time, generates functionally complete and executable code, and achieves performance on par with human-written implementations—establishing a通用, robust, and scalable paradigm for reproducible networking research.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Learning to Search

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web performance, measurement, and characterization
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

Reproducing networking research is a critical but challenging task due to the scarcity of open-source code. While Large Language Models (LLMs) can automate code generation, current approaches lack the generalizability required for the diverse networking field. To address this, we propose RePro, a semi-automated reproduction framework that leverages advanced prompt engineering to reproduce network systems from their research papers. RePro combines few-shot in-context learning with Structured and Semantic Chain of Thought (SCoT/SeCoT) techniques to systematically translate a paper's description into an optimized, executable implementation. The framework operates through a three-stage pipeline: system description extraction, structural code generation, and code optimization. Our evaluation with five state-of-the-art LLMs across diverse network sub-domains demonstrates that RePro significantly reduces reproduction time compared to manual efforts while achieving comparable system performance, validating its effectiveness and efficiency.

Problem

• Reproducing networking research with limited open-source code availability
• Addressing LLMs' lack of generalizability for diverse networking tasks
• Automating translation of paper descriptions into executable implementations

Innovation

• Uses few-shot in-context learning for code generation
• Applies Structured and Semantic Chain of Thought techniques
• Implements three-stage pipeline for system reproduction