Search papers, labs, and topics across Lattice.
This paper identifies a critical failure mode in hard prompt compression known as referential dangling, where independent selection of text leads to the omission of essential context needed for interpretation. The authors demonstrate that at a compression ratio of 0.30, existing hard compressors leave significant gaps in multi-hop question answering tasks, with dangling references occurring in up to 60% of cases. By reinserting missing supporting paragraphs, they achieve a substantial accuracy improvement of 29-34 percentage points, highlighting the necessity for compressors to ensure both relevance and referential completeness.
Hard prompt compressors can leave critical context gaps, leading to a staggering 60% of examples suffering from referential dangling, which severely impacts accuracy in multi-hop question answering.
Hard prompt compression reduces long-context inference cost by independently scoring tokens, sentences, or chunks and retaining the highest-scoring units under a budget. We identify a structural failure in this procedure: independent selection can split dependent evidence pairs, retaining one member while deleting the other. When retained text contains an answer but deleted text defines the entity needed to interpret it, we call the result referential dangling. At a compression ratio of 0.30, Beaver, which ranks coherent chunks using Qwen3-0.6B embeddings, leaves the answer path incomplete in 34-54% of bridge examples across three multi-hop question answering datasets. On a shared HotpotQA bridge set, all six hard compressors we test exhibit dangling at rates up to 60%, and every document in LongBench-v2 Single-Document QA contains at least one dangling reference. On dangling examples evaluated with Qwen3-8B, reinserting the missing supporting paragraph while removing nonsupporting paragraphs to maintain the token budget improves accuracy by 29-34 percentage points (p < 0.0001), recovering at least 88% of the gap to contexts retaining both supporting paragraphs. Stronger answer models do not absorb the loss: on MuSiQue, GPT-5.5 is 8.8 points less accurate on compressed contexts than on contexts retaining both supporting paragraphs. Finally, we train a compact classifier to rank omitted sentences by whether they are needed to interpret retained text and reinsert the top-ranked candidates without support annotations at inference. On HotpotQA with Qwen3-8B, this automatic restoration improves accuracy by 4.7 points while changing the compression ratio only from 0.30 to 0.31. Hard compressors should optimize both relevance and referential completeness.
Tsinghua AI