Search papers, labs, and topics across Lattice.
This paper introduces a framework for automatically annotating reasoning steps in Large Reasoning Models (LRMs) using Bloom's Taxonomy, which categorizes cognitive processes into six levels. By conducting a large-scale analysis, the authors uncover significant similarities and differences in reasoning patterns across various models and tasks, revealing that the type of thinking employed correlates with the correctness of responses. These insights not only enhance our understanding of LRM behavior but also pave the way for actionable improvements in reasoning quality.
Thinking types in LRMs directly correlate with correctness, revealing that cognitive profiling can enhance reasoning performance.
Large Reasoning Models (LRMs) have revolutionized reasoning in LLMs, and the increasing public availability of reasoning traces creates valuable opportunities to study model behavior not only at the surface level but also at the granularity of individual reasoning steps. However, understanding the types of thinking employed during reasoning - which offers critical insights into models' reasoning patterns and enables actionable applications - remains underexplored. To address this gap, we introduce a framework for automatic annotation of reasoning steps through the lens of Bloom's Taxonomy, which classifies thinking into six cognitive levels, such as Remembering, Applying and Evaluating. Using this framework, we perform a large-scale analysis across models and datasets, revealing both similarities and differences in thinking patterns across models and tasks. Moreover, we demonstrate that thinking-type information derived from reasoning traces correlates with correctness, paving the way for improved reasoning. Our findings establish a fine-grained framework for analyzing thinking patterns in LRMs and provide actionable insights for enhancing reasoning quality.
