Search papers, labs, and topics across Lattice.
This study evaluates the effects of reasoning interfaces on the performance of large language models (LLMs) by analyzing paired runs of questions at matched reasoning horizons across two datasets. The findings indicate that a numeric/concision prompt can reduce reasoning traces by 12-17% without significantly compromising accuracy, while a concise instruction can improve accuracy on the MMLU-Pro dataset by 3.8 percentage points at 512 tokens. Additionally, the research reveals that lower-effort, earlier stopping strategies can yield higher accuracy in certain contexts, suggesting a nuanced relationship between reasoning depth and performance outcomes.
Shortening reasoning time can enhance accuracy in LLMs, with a concise instruction boosting performance by nearly 4 percentage points on key benchmarks.
Large language models often reason at length before answering, increasing cost and latency. Prompts and trained settings can shorten this reasoning, but a shorter trace may only show that the model stopped sooner. Here, we evaluate paired runs of the same question at matched reasoning horizons across 198 GPQA Diamond and 500 MMLU-Pro questions. We test a numeric/concision prompt that announces a token limit for Qwen3-14B and the trained effort settings of gpt-oss-20b and -120b. The Qwen prompt shortens reasoning traces by 12-17%, while accuracy changes at matched token limits are small and mixed. A concise/early-answer instruction raises MMLU-Pro accuracy by 3.8 percentage points at 512 tokens, including +2.7 points when both runs are unfinished. Its gain at 2,048 tokens is uncertain. For gpt-oss, candidate-logit answers from completed low- and medium-effort reasoning are 14.5-26.3 points more accurate than matched-horizon high-effort answers. Most of the 512-token advantage comes from lower effort finishing earlier, while differences among unfinished runs are smaller and mixed. Wrong early answers often concentrate probability on the chosen option, so earlier stopping does not uniformly improve probability quality. In these tests, a tight deadline can favor lower effort or a concise instruction, whereas allowing high effort to finish can recover higher final accuracy. Evaluations should report correct completion before a deadline, the answer obtained when a run is stopped, differences among unfinished runs, and probability assigned to the correct answer separately.
