Search papers, labs, and topics across Lattice.
This study investigates how large language models (LLMs) utilize evidence and seek information during decision-making under uncertainty by analyzing their performance in two-armed bandit trials. The researchers found that engaging in inference-time thinking enhanced value-guided actions and reduced noise in choices, while not promoting exploration behaviors typically associated with uncertainty. Notably, thinking length correlated with metacognitive processes, suggesting that LLMs can improve decision-making by effectively leveraging available evidence without necessarily seeking out additional information.
Thinking enhances decision-making in LLMs by improving action based on current evidence, but doesn鈥檛 drive them to seek more information.
Inference-time thinking improves the performance of large language models, but aggregate outcomes do not reveal whether models use available evidence more effectively or seek information that could improve future decisions. We distinguish these responses by measuring action preference, thinking length, and reported confidence under matched uncertainty. Ten open-weight models completed matched horizon-style two-armed bandit trials in thinking and non-thinking modes. A cognitive model separated value-guided action and uncertainty-independent choice noise from two behavioral signatures of exploration: a UCB-like preference for the less-known arm and Thompson-like choice variability that increases with total uncertainty. On average, thinking strengthened value-guided action and reduced uncertainty-independent choice noise, without producing UCB-like exploration or strengthening Thompson-like exploration. Outside action, the information-imbalanced history condition, which also displayed more observations than the matched balanced condition, was associated with greater thinking length. Reported confidence became more sensitive to decision difficulty and more strongly associated with chosen task evidence. We interpret these thinking-length and reported-confidence patterns as consistent with metacognitive control and metacognitive monitoring, respectively, without establishing either process. Decoder sweeps, especially temperature, altered choice noise and thinking length but did not reproduce the joint cross-output pattern. In this controlled decision setting, thinking improved how models acted on current evidence, while neither measured signature supported a shift toward a more information-seeking policy.
