Search papers, labs, and topics across Lattice.
This paper investigates the phenomenon of task insensitivity in large language models, revealing that these models often default to learned patterns rather than adapting to new, distinct tasks, even when faced with semantically corrupted instructions. The authors identify a training-time attention drift away from task-specific tokens towards local observations as a contributing factor to this issue. To address this, they introduce Task-Perturbed NLL Optimization, a contrastive regularizer that enhances task sensitivity and out-of-distribution generalization, leading to improved performance without sacrificing attention stability to task instructions.
Task insensitivity in language models leads to significant failures in OOD generalization, but a novel optimization technique can reverse this trend.
Large language models can serve as capable long-horizon agents, but their out-of-distribution (OOD) generalization remains weak. We identify a key source of this failure as task insensitivity: when faced with similar but distinct tasks, models might apply patterns learned during training and fail to solve the task at hand. We show that models often continue with actions aligned with the original task even when the instruction is semantically corrupted and cannot be directly answered. We further find that, when we replace the task description in a trained prompt with another similar but distinct task, the model may still output the same action. This behavior is accompanied by a consistent training-time attention drift away from task tokens and toward local observations, suggesting an optimization bias toward shortcuts. To mitigate this problem, we propose Task-Perturbed NLL Optimization, a lightweight contrastive regularizer that explicitly encourages action dependence on the task instruction. Extensive evaluations show that our intervention improves task sensitivity and OOD generalization while preserving more stable attention to task tokens.
