Search papers, labs, and topics across Lattice.
This paper investigates the impact of different chunking strategies on retrieval-augmented code completion, testing Function, Declaration, Sliding Window, and cAST methods across various retrievers, generators, and parameter settings. The study reveals that Function chunking consistently underperforms other strategies, while Sliding Window and cAST chunking achieve Pareto-optimal performance. They also find that cross-file context length is a more significant factor than chunk size in improving code completion quality.
Turns out, chunking code by function is the *worst* way to do retrieval-augmented code completion.
Retrieval-augmented generation (RAG) pipelines for code completion rely on chunking to segment source files into retrievable units, yet chunking strategies are typically adopted without empirical justification, and practitioner recommendations are notably inconsistent. We present a controlled empirical study isolating the effect of chunking on code completion quality by crossing four representative strategies (Function, Declaration, Sliding Window, and cAST) with four retrievers, five generators, and nine parameter configurations on two benchmarks (RepoEval and CrossCodeEval), totaling 864 experimental settings. Our results reveal that chunking strategy has a statistically significant effect on RAG-based code completion. Contrary to intuition, chunking based on functions underperforms all other strategies by 3.57--5.64 percentage points on RepoEval (Cliff's delta = -1.0), while the remaining chunking strategies perform comparably. Our further analysis demonstrates that this observation holds across all retriever--generator combinations. We also find that cross-file context length is the dominant parameter: doubling from 2,048 to 8,192 tokens yields up to 4.2 percentage points of improvement, whereas chunk size has a weaker, non-monotonic effect. On the cost--quality Pareto front, Sliding Window and cAST dominate both benchmarks; Function chunking is never Pareto-optimal.
