Search papers, labs, and topics across Lattice.
This paper introduces ACToR, an adaptive critical token-aware retrieval framework designed to enhance repository-level code generation by identifying and retrieving context for critical tokens during the autoregressive generation process. By focusing on these decisive positions, ACToR improves the accuracy of code synthesis, mitigating the risk of functional failure due to incorrect token generation. Experimental evaluations demonstrate that ACToR outperforms existing state-of-the-art methods, achieving significant relative improvements of 8.4% and 15.4% on the RepoExec and CoderEval benchmarks, respectively.
Critical tokens can make or break code generation, and ACToR ensures they receive the precise context needed to succeed.
The repository-level code generation task requires synthesizing code that satisfies task requirements while remaining consistent with the target repository context. Since real-world repositories often exceed the input length limits of LLMs, existing approaches commonly adopt retrieval-augmented generation (RAG) to provide repository-specific context. Despite improving repository-context retrieval, existing methods typically provide context as task-level support, without explicitly identifying the critical tokens that require fine-grained repository context during generation. During the autoregressive generation process of LLMs, errors often concentrate at a small number of decisive positions: once such tokens are generated incorrectly, subsequent code may follow an incorrect semantic path and eventually lead to functional failure. We refer to these positions as"critical tokens". In this paper, we propose ACToR, an adaptive critical token-aware retrieval framework for repository-level code generation. ACToR identifies critical tokens during generation and triggers targeted retrieval on demand to provide repository context at these decisive positions. In addition, we design a position-aware weighting method for dense retrievers to prioritize context that is more informative for generation. We evaluate ACToR on two representative repository-level benchmarks, RepoExec and CoderEval. Experimental results show that ACToR consistently outperforms state-of-the-art methods, achieving relative improvements of 8.4% on RepoExec and 15.4% on CoderEval. Beyond performance gains, we systematically quantify the impact of critical tokens, revealing their central role in major generation failures and highlighting the necessity of targeted retrieval strategies. We provide the code and data at https://github.com/DeepSoftwareAnalytics/ACToR.
