Search papers, labs, and topics across Lattice.
This paper identifies "cliff tokens," specific tokens that trigger significant drops in performance during mathematical reasoning tasks in large language models (LLMs). By employing a one-sided two-proportion z-test, the authors demonstrate that removing these cliff tokens can dramatically improve model accuracy, achieving pass rates from 64% to 100% across various benchmarks. Additionally, they introduce a taxonomy of cliff types based on their probabilistic characteristics, which aids in understanding and optimizing LLM performance in reasoning tasks.
Identifying cliff tokens reveals that a single token can drastically shift LLM performance, with targeted removal leading to near-perfect accuracy in mathematical reasoning tasks.
Large language models (LLMs) reach high accuracy in mathematical reasoning, but individual traces on the same problem diverge; some arrive at the correct answer while others fail. Prior work analyzes failure at the step, chunk, or sentence level, or at tokens where failure has already occurred. Neither identifies the precise token that triggers the shift toward failure. We introduce the cliff token, a token where the token-wise potential drops significantly under an adaptive threshold that scales with the local token-wise potential, based on a one-sided two-proportion z-test. Across seven models and three mathematical reasoning benchmarks (GSM1K, MATH500, AIME 2025), cliff tokens act as failure triggers; deleting the first cliff token and resampling recovers pass@64 to 1.0, while keeping it limits recovery to between 0.71 and 1.00. We further introduce a cliff taxonomy of deterministic, uncertain, and sampled-off cliffs, defined by greedy choice and token entropy. Each type has distinct probabilistic characteristics, and the taxonomy generalizes across model scales. Finally, we validate the taxonomy via single-token preference optimization at cliff positions (Cliff-DPO). Trained on GSM8K, Cliff-DPO improves accuracy across benchmarks by up to +6.6. Optimizing at uncertain and sampled-off cliffs improves reasoning, while deterministic cliffs do not.
