Search papers, labs, and topics across Lattice.
The paper investigates the "Lost in Conversation" phenomenon, where LLMs suffer significant performance degradation when tasks are revealed incrementally across multiple turns, identifying unreliability as the primary driver. To address this, they propose SeDT (Sentence-transformer Decision-Transformer), a training-free inference-time method that annotates each conversation turn with a cumulative relevance score derived from semantic, lexical, and positional signals. SeDT substantially recovers performance lost in conversation by highlighting relevant past turns, achieving gains up to +37.7% in mean performance and reducing unreliability across various LLMs and generation tasks.
LLMs forget critical context in multi-turn conversations not because they lack aptitude, but because they can't distinguish important turns from irrelevant ones – and a simple relevance-weighted history fixes it.
Large language models (LLMs) achieve impressive performance when a task is fully specified in a single turn, yet the same models lose up to 39% of that performance when the identical task is revealed incrementally across multiple turns, a phenomenon documented at scale as Lost in Conversation. Crucially, this collapse is almost entirely a reliability failure; the best case, the aptitude only falls 16%, while the unreliability more than doubles (+112%). We argue that the root cause is structural, a flat conversation history assigns equal implicit weight to every prior turn, giving the model no signal to distinguish a critical constraint from incidental dialog. We present SeDT Sentence-transformer Decision-Transformer, a training-free inference-time method that resolves this by importing return-to-go conditioning from offline reinforcement learning. SeDT annotates each conversation shard with a cumulative relevance score derived from three complementary semantic, lexical, and positional signals and presents the full annotated history to the model at the final turn, without weight changes, without training data, and without discarding context. Evaluated on the Lost-in-Conversation benchmark in three LLMs and three generation tasks, SeDT outperforms the sharded baseline in all nine model-task combinations, with gains up to +37.7% in mean performance P and simultaneous reductions in unreliability in seven of the nine combinations. In short, telling the model which past turns matter is sufficient to substantially recover the performance lost in conversation.
