Search papers, labs, and topics across Lattice.
The authors systematically evaluate the reliability of LLM agents' self-reported progress across execution lifecycles using the $\tau^2$-bench benchmark and a novel checkpointed testbed called StageIF. Across evaluated models, progress tracking accuracy fluctuates sharply by execution phase: standard deployed models lose calibration mid-task before recovering, whereas frontier models resolve mid-task errors only to become pathologically conservative at completion. These stage-dependent failure modes reveal that autonomous agent orchestration frameworks cannot safely rely on model self-reflection alone to govern control flow and termination.
Frontier LLM agents still struggle to judge their own completion: while older models lose tracking accuracy mid-task, the newest generation reliably monitors intermediate execution only to freeze conservatively at the finish line.
Recent large language models can emit task-progress signals that agent frameworks use to decide whether a task should continue or stop, yet whether a model can reliably report its task progress at every stage of a task, and where and how its reports fail, has not been studied systematically. We evaluate this ability on the public benchmark $\tau^2$-bench and on StageIF, a controlled testbed in which reporting checkpoints are placed across the task's lifecycle. Both settings require reports at multiple task stages. We find that reporting reliability depends on the stage a task has reached, and that almost every deployed model we test is reliable at some stages and unreliable at others. Where reporting breaks down is not the same everywhere. Most deployed models lose accuracy once work is under way and recover once the task is done. The newest generation closes that mid-task drop and instead grows conservative at the finish line. Our study exposes a capability gap in task-progress reporting and provides an evaluation protocol that spans the whole course of task execution for this ability on which agent operation depends. The findings indicate that agent frameworks should not control task flow on the strength of the model's state reports alone.
