Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates language models' ability to selectively leverage external guidance, termed "thinking outside the box," by introducing Box^2-Bench to isolate guidance reliability. Frontier models demonstrate benefits from dependable workflows but struggle with misleading or unreliable inputs. Counterfactual supervised fine-tuning and outcome-based reinforcement learning are explored as training strategies to enhance this selective reliance, extending its benefits to peer correction and corrupted memory robustness.
Frontier LLMs can be trained to selectively ignore unreliable external guidance, a capability crucial for robust agentic behavior beyond simple task accuracy.
Agent harnesses often improve language models with human-designed workflows, but as models grow more capable, unreliable guidance can increasingly constrain their execution. We call the ability to benefit from useful guidance while overriding unreliable guidance thinking outside the box. We introduce Box^2-Bench, which holds the model and task fixed while varying workflow reliability to isolate how models regulate their reliance on guidance. On Box^2-Bench, frontier models often benefit from reliable guidance but remain vulnerable when it is misleading or becomes unreliable. To test whether this capability can be learned, we train two open-weight models using bad workflows, reserving good workflows for evaluation. We explore two complementary training strategies: counterfactual supervised fine-tuning improves robustness, while outcome-based reinforcement learning can shift the balance toward greater use of helpful workflows. We further find that this behavior extends beyond workflows to other forms of external information, improving peer correction and robustness to corrupted memory. Together, our results identify selective reliance on fallible external information as a dimension of agent reliability not captured by task performance alone.