Search papers, labs, and topics across Lattice.
This paper investigates the hidden-state trajectories of LLMs during multi-turn reasoning, focusing on how temporal curvature and variance slope can signal the stability of task-relevant information. By analyzing four tasks across three LLMs, the authors reveal that these geometric signals can predict the success of reasoning episodes before they conclude, highlighting the importance of action-dependent separability in reasoning chains. The findings show a significant improvement in task success rates from 24.1% to 39.6% while simultaneously reducing token costs by 11.2%, demonstrating the potential for more efficient multi-turn interactions.
Trajectory geometry can predict reasoning success in LLMs, boosting task performance while cutting costs.
LLM agents need to sustain goal-consistent reasoning across long multi-turn interactions under strict resource constraints. However, as the multi-turn context accumulates, it can destabilize the underlying LLM's internal representation of task-relevant information from earlier turns, blurring the boundary between constructive reasoning and representation drift. We formulate multi-turn reasoning as a hidden-state trajectory of the underlying LLM that is characterized via two complementary signals: temporal curvature that captures the directional consistency of turn-to-turn updates, and variance slope which measures the expansion or contraction of the exploration space. Across four tasks and three underlying LLMs, we observed that these geometric signals distinguish between correct and incorrect episodes prior to completion. We further decompose each episode into three-action chains formed from four actions (Read, Write, Respond, Transfer) and show that separability is action-dependent, with different signals distinguishing various chain patterns. Our experiments demonstrate that trajectory geometry can identify critical turns in the reasoning process, increasing task success rates on $蟿$-Bench from 24.1% to 39.6% while reducing token cost by 11.2%.
