Search papers, labs, and topics across Lattice.
This paper introduces Second Thought, an innovative inference framework that allows LLM agents to engage in parallel reasoning during the Action and Observation phases of the ReAct paradigm. By forking auxiliary reasoning branches immediately after the Thought phase, the framework reduces the average turn count and main thread decoding time significantly, achieving up to 43% reduction in decoding while maintaining or improving performance metrics like Pass@1. The results across multiple benchmarks demonstrate that Second Thought enhances efficiency without sacrificing reasoning quality, offering a compelling advancement in agent design.
Parallel reasoning in LLM agents can cut decoding time by up to 43% while maintaining performance, reshaping agent efficiency.
LLM agents in the ReAct paradigm alternate between reasoning, acting, and observing, but deliberate reasoning is confined to the Thought phase: while the agent serializes an action and waits for the environment, its reasoning is frozen. We identify this recurring interval for Action and Observation as a reasoning idle window and ask whether it can host additional reasoning in parallel that serves future turns. Therefore, we propose Second Thought, a training-free inference framework that forks four auxiliary branches the instant each Thought phase concludes, decodes them concurrently with the main loop, and merges the generated thoughts back when the environment observation arrives. In this way, Second Thought relocates the added reasoning off the main thread's sequential decoding path. Across three agentic benchmarks and three reasoning LLMs, Second Thought lowers the average turn count in all nine (model,benchmark) pairs and reduces main thread decoding in six of them by up to 43% (roughly 20% on average among those settings), while leaving it essentially unchanged in a seventh; Pass@1 shows no significant change in seven of nine pairs and the two significant differences are +12.4 and +10.2 points. Against a compute-matched control that forces an equivalent budget onto the main thread's own reasoning, it attains strictly higher Pass@1 with 1.3 to 3.2 less sequential decoding in all four settings where the control applies.
