LLM Agents Can See Code Repositories

2026-06-11

AI Summary

This work addresses the limitation of current large language model–based coding agents, which treat code repositories as plain text and overlook the visual structures—such as directory hierarchies and dependency graphs—that human developers rely on to navigate complex codebases. The study presents the first empirical validation of incorporating visual structure graphs, combining directory trees and dependency relationships, as an auxiliary modality in repository-level program repair tasks. It proposes a hybrid interaction paradigm that fuses textual and visual information. Experiments across four state-of-the-art multimodal large language models demonstrate that this approach maintains or improves repair accuracy while reducing input token usage by up to 26% compared to purely textual or purely visual inputs, with particularly pronounced gains in fault localization and autonomous exploration scenarios.

Abstract

Coding agents powered by large language models have demonstrated strong performance on software engineering tasks. Yet most agents consume repositories almost entirely as text, which differs from how human developers use visual structure such as folder hierarchies and dependency relationships to orient themselves in large codebases. With multimodal large language models (MLLMs), it is an open question whether agents can effectively benefit from visual representations of repositories. This paper presents the first systematic empirical study of visual repository representations for LLM-based agents on repository-level issue resolution. We evaluate four recent multimodal models. Our results show that a strictly vision-only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries. In contrast, integrating visual graphs of repository structure as a supplementary modality alongside standard text interfaces helps agents understand structure more efficiently: input token consumption decreases by up to 26% while issue-resolution accuracy is maintained or improved. Visualization is most useful during fault localization and when the agent autonomously controls exploration depth. These findings point to a practical hybrid text-and-vision design for next-generation coding agents.

Problem

• LLM agents
• code repositories
• visual representation
• multimodal models
• issue resolution

Innovation

• multimodal LLM
• code repository visualization
• coding agents
• hybrid text-vision interface
• repository-level issue resolution