Search papers, labs, and topics across Lattice.
This paper introduces TopoCompress, a novel framework for compressing long contexts in large language models without requiring additional training or alignment. By leveraging a hybrid graph structure that connects semantically similar spans and propagates relevance scores, TopoCompress effectively selects coherent semantic spans for compression. The approach demonstrates significant efficiency, achieving comparable performance to existing methods while using a fourfold smaller compression budget and reducing compression time by 1.41 times.
Achieving 4x smaller compression budgets without sacrificing performance, TopoCompress redefines long-context handling in language models.
Long-context compression is essential for reducing the cost and latency of large language model inference. However, existing methods can fragment important evidence, require additional training or alignment, and often depend on the target model for effective compression. We introduce TopoCompress, a training-free and model-agnostic framework that compresses long contexts by selecting coherent semantic spans. TopoCompress first scores each span using dense and lexical query relevance together with semantic acceleration. It then constructs a hybrid graph that connects spans based on semantic similarity and sequential adjacency, and propagates the query-guided relevance scores over the graph. Across five long-context tasks-HotpotQA, 2WikiMQA, MuSiQue, Qasper, and MultiFieldQA-en-TopoCompress consistently outperforms strong compression baselines. Notably, TopoCompress achieves performance comparable to the strongest baseline while using a 4x smaller compression budget, and provides a 1.41x smaller compression time over the fastest baseline.
