RAT: RunAnyThing via Fully Automated Environment Configuration

2026-04-25

AI Summary

This work addresses the challenge of highly manual and non-generalizable environment configuration in repository-level software engineering tasks by introducing RAT, the first language-agnostic framework for fully automated repository setup. RAT establishes an end-to-end pipeline comprising semantic initialization, task planning, invocation of specialized tools, and robust sandbox construction. To evaluate such systems realistically, the authors also release RATBench, the first benchmark reflecting the true distribution and heterogeneity of real-world code repositories. Experimental results demonstrate that RAT significantly outperforms strong existing baselines on RATBench, achieving an average 29.6% improvement in Environment Setup Success Rate (ESSR). This advance overcomes prior limitations that relied on predefined artifacts or were confined to specific programming languages.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Planning, Routing, and Scheduling: Planning with Language Models
• Intelligent Robots: Multi-Robot Systems

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

Automating repository-level software engineering tasks is a foundational challenge for autonomous code agents, largely due to the difficulty of configuring executable environments. However, manual configuration remains a labor-intensive bottleneck, necessitating a transition toward fully automated environment configuration. Existing approaches often rely on pre-defined artifacts or are restricted to specific programming languages, limiting their applicability to real-world repositories. In this paper, we first propose RAT (RunAnyThing), a language-agnostic framework for automated environment configuration on arbitrary repositories. RAT features a multi-stage pipeline that integrates semantic initialization, a planning mechanism, specialized toolset, and a robust sandbox for configuration. Furthermore, to enable rigorous evaluation, we propose RATBench, a benchmark that reflects the the distribution and heterogeneity of real-world repositories. Extensive experiments demonstrate that RAT achieves state-of-the-art performance, improving the Environment Setup Success Rate (ESSR) by an average of 29.6% over strong baselines.

Problem

• automated environment configuration
• repository-level software engineering
• executable environments
• language-agnostic
• autonomous code agents

Innovation

• automated environment configuration
• language-agnostic
• repository-level software engineering
• multi-stage pipeline
• sandbox execution