Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of hidden-state probes for localizing reasoning failures in large language models, finding that standard probes trained on final-answer correctness conflate distinct failure modes. By introducing Monte Carlo rollouts and analyzing different probe training designs, the research reveals that current probes fail to accurately distinguish between inherent capability limitations, recoverable trace errors, and single-step failures. The key result is that even probes designed to predict the failing step often perform no better than simple heuristics like trace length, highlighting significant limitations in current interpretability methods for complex reasoning.
LLM reasoning failures are often misdiagnosed by standard probes, with even specialized probes failing to pinpoint the exact faulty step more reliably than trace length.
Verifying the final answer of a long reasoning trace may show that the trace is wrong but it does not distinguish three cases. The problem may lie beyond the model's capability, another attempt would succeed, or one step in an otherwise recoverable trace made the trace fail. Standard hidden-state probes are trained on final-answer correctness, a label that conflates these cases. Monte Carlo rollouts distinguish the cases and locate the failing step at additional inference cost. We test the hidden states of 4B and 8B reasoning models for the failure type and the failing step and compare each probe with the strongest baseline that uses no hidden states. On incorrect 8B traces, the confidence probe, a calibrated correctness probe at a midpoint cutoff, misses 81% of cases in which a single step makes the trace unrecoverable and 47% of cases in which another attempt would succeed. A probe with a different training design over the same states reverses the ordering of these two kinds of failure, so either ordering can be read from those states. At both scales, no single tested probe locates the failing step more accurately than a rule based only on trace length, and at 4B a probe trained to predict that step fails to outperform this rule in any of 10 runs on identical traces. The confidence probe therefore indicates that an answer is wrong, but its failure-type ordering depends on its training design.