AgentLogs: A Dataset for Opening the Black Box of GitHub's Cloud Agent

2026-08-29

AI Summary

为了解决生成式AI软件工程代理工作过程不透明的问题，研究者通过创建AgentLogs数据集记录GitHub云代理的详细活动日志。

Abstract

Generative AI-based software engineering agents are becoming routine contributors to real-world software projects. On GitHub, developers can assign tasks to the Copilot cloud agent, which autonomously explores the repository, edits code, runs commands, and opens or reviews pull requests, producing a detailed log of every step along the way. While existing datasets capture outcomes of agent contributions, such as agent-authored pull requests, the process by which agents produce these contributions remains largely unexplored. To address this gap, we introduce AgentLogs, a large-scale dataset of agent activity on GitHub. AgentLogs comprises 307,416 agent tasks and 549,239 agent sessions in 35,810 of the 1,812,362 popular public repositories that we scanned, together with 64,255,174 session log entries that record each agent run step by step, including prompts, intermediate reasoning, tool calls (e.g., file edits, git operations, and GitHub interactions), and token usage. By exposing not only what agents contribute but also how they work, AgentLogs enables research on agent behavior, efficiency and cost, task formulation, failure modes, and human-agent collaboration in agentic software engineering.

Problem

• Agent Behavior
• Software Engineering
• GitHub
• AI-based Agents
• Dataset

Innovation

• AgentLogs
• GitHub Cloud Agent
• Software Engineering
• Dataset
• Agent Activity