Search papers, labs, and topics across Lattice.
This study investigates the impact of visual representations of code repositories on the performance of large language model (LLM) agents in software engineering tasks. By systematically evaluating four multimodal models, the authors find that while a vision-only approach hampers accuracy and increases token costs, integrating visual graphs with text interfaces enhances agents' understanding of repository structures. The results indicate that this hybrid approach can reduce input token consumption by up to 26% while maintaining or improving issue-resolution accuracy, particularly aiding in fault localization and exploration depth control.
Integrating visual graphs with text interfaces allows LLM agents to reduce token consumption by 26% while enhancing their issue-resolution accuracy.
Coding agents powered by large language models have demonstrated strong performance on software engineering tasks. Yet most agents consume repositories almost entirely as text, which differs from how human developers use visual structure such as folder hierarchies and dependency relationships to orient themselves in large codebases. With multimodal large language models (MLLMs), it is an open question whether agents can effectively benefit from visual representations of repositories. This paper presents the first systematic empirical study of visual repository representations for LLM-based agents on repository-level issue resolution. We evaluate four recent multimodal models. Our results show that a strictly vision-only setup degrades accuracy and increases token cost, because agents lack sufficient symbolic detail and compensate with repeated visual queries. In contrast, integrating visual graphs of repository structure as a supplementary modality alongside standard text interfaces helps agents understand structure more efficiently: input token consumption decreases by up to 26% while issue-resolution accuracy is maintained or improved. Visualization is most useful during fault localization and when the agent autonomously controls exploration depth. These findings point to a practical hybrid text-and-vision design for next-generation coding agents.
