Search papers, labs, and topics across Lattice.
SproutRAG introduces an attention-guided hierarchical retrieval-augmented generation framework that organizes sentence-level chunks into progressively larger, semantically coherent units, addressing the trade-offs in retrieval granularity and contextual coherence. By leveraging learned inter-sentence attention to construct a binary chunking tree, it eliminates the need for costly external LLM calls and lossy summarization. Experiments show that SproutRAG achieves a 6.1% improvement in information efficiency over the strongest baseline across diverse benchmarks in scientific, legal, and open-domain contexts.
SproutRAG achieves a 6.1% boost in information efficiency by intelligently structuring document chunks without relying on external LLMs or lossy summarization.
Retrieval-augmented generation (RAG) systems must balance retrieval granularity with contextual coherence, a challenge that existing methods address through LLM-guided chunking, single-level context expansion, or hierarchical summarization. These approaches variously depend on costly LLM calls during indexing or retrieval, limit context aggregation to a single granularity level, or introduce information loss through summarization. We present SproutRAG, an attention-guided hierarchical RAG framework that addresses this trade-off by organizing sentence-level chunks into progressively larger but semantically coherent units, using learned inter-sentence attention to construct a binary chunking tree. Unlike prior approaches that rely on external LLMs, fixed context expansion, or lossy summarization, SproutRAG learns which attention heads and layers best capture semantic document structure, enabling multi-granularity retrieval without additional LLM calls or compressed summaries. At retrieval time, SproutRAG uses hierarchical beam search to retrieve candidates at multiple granularities, capturing multi-sentence relevance beyond flat retrieval. The framework is trained end-to-end with a joint objective that improves both embeddings and tree structure. Experiments across four benchmarks spanning scientific, legal, and open-domain settings demonstrate that SproutRAG improves information efficiency (IE) by 6.1% on average over the strongest baseline. Code is available on https://github.com/AmirAbaskohi/SproutRAG.
