Configuring Agentic AI Coding Tools: An Exploratory Study

2026-02-16

AI Summary

Current agent-based AI programming tools lack a systematic understanding of their configuration mechanisms, hindering effective usage and evolution. This study addresses this gap by empirically analyzing 2,926 GitHub repositories to construct the first taxonomy of configuration approaches, encompassing Context Files, Skills, and Subagents. Through large-scale repository mining, configuration content analysis, and cross-tool comparison, we uncover significant differences in configuration practices among mainstream tools such as Claude Code and GitHub Copilot. Our findings reveal that Context Files dominate current usage, while advanced mechanisms remain underutilized and largely consist of static instructions. Notably, Claude Code users exhibit the richest configuration patterns, and AGENTS.md is emerging as a de facto standard for cross-tool interoperability. These results establish an empirical foundation for future research on configuration strategy evolution and its impact on tool performance.

Technology Category

• Cognitive Modeling & Cognitive Systems: Agent Architectures
• Multiagent Systems: Modeling other Agents
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Search and Retrieval-Augmented AI: Agentic search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Agentic AI coding tools with autonomous capabilities beyond conversational content generation increasingly automate repetitive and time-consuming software development tasks. Developers can configure these tools through versioned repository-level artifacts such as Markdown and JSON files. In this paper, we present a systematic analysis of configuration mechanisms for agentic AI coding tools, covering Claude Code, GitHub Copilot, Cursor, Gemini, and Codex. We identify eight configuration mechanisms and, in an empirical study of 2,926 GitHub repositories, examine whether and how they are adopted. We then explore Context Files, Skills, and Subagents, that is, three mechanisms available across tools, in more detail. Our findings reveal three trends. First, Context Files dominate the configuration landscape and are often the sole mechanism in a repository, with AGENTS$.$md emerging as an interoperable standard across tools. Second, advanced mechanisms such as Skills and Subagents are only shallowly adopted: most repositories define only one or two artifacts, and Skills predominantly rely on static instructions rather than executable workflows. Third, distinct configuration cultures are forming around different tools, with Claude Code users employing the broadest range of mechanisms. These findings establish an empirical baseline for longitudinal and experimental research on how configuration strategies evolve and affect agent performance as agentic AI coding tools mature.

Problem

• Agentic AI
• Configuration Mechanisms
• AI Coding Tools
• Software Development Automation
• GitHub Repositories

Innovation

• Agentic AI
• Configuration Mechanisms
• Context Files
• Skills
• Subagents