Search papers, labs, and topics across Lattice.
This study evaluates token-efficient retrieval methods for analyzing transactional legal documents, comparing full-corpus injection with two structured retrieval approaches: embedding retrieval (NAVEMBED) and LLM navigation (NAVINDEX). The results show that both NAVEMBED and NAVINDEX achieved comparable semantic retrieval performance to full injection while significantly reducing token usage, with NAVINDEX demonstrating a 1.61x smaller token footprint and 25% lower cost. The findings suggest a closed-form caching-crossover rule that indicates when cached injection remains cost-effective, providing a valuable framework for optimizing legal document analysis in LLMs.
Token-efficient retrieval methods can match full-corpus injection performance while slashing token usage and costs, revolutionizing how we analyze legal documents with LLMs.
Answering questions over a set of transactional legal documents is most simply done by injecting the whole corpus into the LLM's context window on every query. That baseline maximises retrieval recall, but its token footprint scales with the corpus rather than the question, and long-context degradation scales with it. We report what it took to replace full-corpus injection in a legal-document analysis system, comparing it against two structured retrieval modes over our proprietary structure-aware chunking: embedding retrieval (NAVEMBED) and LLM navigation over a compact structured index (NAVINDEX). On a 20-question benchmark with verified ground-truth answers, a position-bias-controlled, reference-anchored pairwise judge scored semantic retrieval with reranking tied with injection on 16 of 18 document-bound questions (injection preferred on 2) while attending to 17.3x fewer input tokens (a general-text-embedding (GTE) configuration reaches 29.9x at a lower tie rate); both modes were judged tied on the 2 out-of-scope controls. NAVINDEX was judged tied on all 18 at a 1.61x smaller total token footprint, a ~56x smaller answering context, and 25% lower dollar cost. We derive a closed-form caching-crossover rule: cached injection is cheaper in dollars only while the corpus stays below roughly ten times the retrieval payload. Scope and uncertainty are quantified in Section 8.
