Towards an Appropriate Level of Reliance on AI: A Preliminary Reliance-Control Framework for AI in Software Engineering

2026-04-12

AI Summary

This study addresses the dual risks of over-reliance and under-reliance on AI tools—such as large language models—that can impair developers’ long-term capabilities and productivity. Through semi-structured interviews with 22 software developers and subsequent thematic analysis, the research introduces a novel AI dependency regulation framework centered on “control levels.” This framework systematically characterizes patterns of AI dependency, delineates two forms of dependency imbalance along with their associated risks, and operationalizes dependency intensity into distinct, actionable control levels. By doing so, it provides a theoretical foundation for fostering responsible and effective use of AI in software development and offers actionable insights for the design of future AI-assisted development tools and related policy formulation.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Humans and AI: Planning and Decision Support for Human-Machine Teams
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

How software developers interact with Artificial Intelligence (AI)-powered tools, including Large Language Models (LLMs), plays a vital role in how these AI-powered tools impact them. While overreliance on AI may lead to long-term negative consequences (e.g., atrophy of critical thinking skills); underreliance might deprive software developers of potential gains in productivity and quality. Based on twenty-two interviews with software developers on using LLMs for software development, we propose a preliminary reliance-control framework where the level of control can be used as a way to identify AI overreliance and underreliance. We also use it to recommend future research to further explore the different control levels supported by the current and emergent LLM-driven tools. Our paper contributes to the emerging discourse on AI overreliance and provides an understanding of the appropriate degree of reliance as essential to developers making the most of these powerful technologies. Our findings can help practitioners, educators, and policymakers promote responsible and effective use of AI tools.

Problem

• AI reliance
• overreliance
• underreliance
• software engineering
• Large Language Models

Innovation

• reliance-control framework
• AI overreliance
• Large Language Models
• software engineering
• human-AI interaction