Search papers, labs, and topics across Lattice.
This study investigates the interaction between document-specific LoRA adapters and KV cache memory in document-level question answering. By progressively evicting KV states, the authors demonstrate that while LoRA contributes minimally when the KV cache is intact, it significantly enhances performance—recovering 13-21 ROUGE-L points—when document context is aggressively compressed. The findings suggest that LoRA should be viewed as a decoding-time parametric memory rather than a document encoder, with QA-style supervision yielding stronger adapters than traditional next-token prediction methods.
Document LoRA can recover up to 21 ROUGE-L points when context is scarce, redefining its role in memory architecture for question answering.
Parametric retrieval augmentation encodes document information into lightweight, document-specific modules such as LoRA adapters, reducing the need to include all evidence as input context. However, it remains unclear how this parameter-side memory interacts with context-side memory stored in the KV cache. We study this interaction in document-level question answering by progressively evicting document key-value states and measuring when a document LoRA contributes beyond the retained context. We find that document LoRA adds little when the KV cache is largely intact, but becomes increasingly useful under aggressive compression, recovering 13-21 ROUGE-L points when no document context remains. The gain is largest when the base model encodes the document, and the adapter is applied only during answer generation, suggesting that document LoRA is better understood as decoding-time parametric memory than as a document encoder. Finally, QA-style supervision produces substantially stronger adapters than raw-context next-token-prediction. These results position document LoRA as a complementary memory channel whose value emerges precisely when context-side evidence is scarce.
Apple ML