Search papers, labs, and topics across Lattice.
This study investigates the impact of task-irrelevant context on the performance of large language models (LLMs), revealing that while aggregate accuracy remains stable, individual predictions can fluctuate significantly. By introducing semantically meaningless pseudo-words, the authors demonstrate that these irrelevant contexts can lead to both performance degradation and improvement on specific examples, highlighting a hidden instability within state-of-the-art models. The findings underscore the importance of evaluating model reliability on a per-example basis, as aggregate metrics may obscure critical vulnerabilities.
Aggregate accuracy masks significant prediction instability, with irrelevant context causing both performance drops and gains on individual examples.
As large language models (LLMs) grow more capable, they are increasingly deployed in context-rich settings where task inputs are often accompanied by long, partially irrelevant context. In a controlled setting, we find that state-of-the-art models often appear robust to task-irrelevant context at the aggregate level: prepending it to benchmark questions causes little change in overall accuracy. This aggregate stability, however, masks significant per-example instability. Even semantically meaningless pseudo-words, formed by randomly combining characters, can markedly shift model predictions on a small fraction of examples, degrading performance on some while improving it on others. This two-sided effect holds consistently across a wide range of models and datasets, yet the affected examples are largely model-specific. We further show that this instability is modulated by context type, context length, test-time compute, and model development stage. Together, our findings reveal context-induced tail risks concealed by aggregate accuracy, motivating per-example reliability evaluation of language models.
