Search papers, labs, and topics across Lattice.
This study investigates the extrinsic and intrinsic properties that enhance reasoning capabilities in large language models (LLMs) when using a Code Interpreter (CI). By analyzing crucial tokens and cognitive behaviors such as verification and backtracking, the authors find that stronger CI reasoning models exhibit these properties more frequently, leading to improved performance in specific reasoning tasks. The research highlights how incorporating code-specific crucial tokens during inference and cognitive behaviors during training can significantly enhance model performance while also identifying limitations in certain models.
Stronger reasoning in LLMs using Code Interpreters hinges on the strategic use of crucial tokens and cognitive behaviors, revealing both potential and constraints in model performance.
Reasoning with a Code Interpreter (CI) has emerged as an effective paradigm for enhancing the reasoning capabilities of large language models (LLMs) through executable computation and iterative verification. Despite its growing adoption, the behavioral properties underlying effective code reasoning remain largely underexplored. In this work, we investigate code reasoning from two distinct perspectives inspired by prior studies of natural language reasoning: extrinsic properties, represented by crucial tokens, and intrinsic properties, represented by code-specific cognitive behaviors. Across multiple LLMs, we find that stronger CI reasoning models consistently exhibit a higher prevalence of crucial tokens and cognitive behaviors, particularly verification, backtracking, and backward chaining. Building on these observations, we examine how these properties can be leveraged during both inference and training. At inference time, appending code-specific crucial tokens improves performance on several reasoning capabilities, including mathematical, ordering, and optimization, while yielding limited benefits elsewhere. At training time, augmenting a state-of-the-art framework with code-specific cognitive behaviors improves supervised fine-tuning and reinforcement learning performance in two of three evaluated models. Further analysis shows that these behaviors reduce overthinking in incorrect responses and improve token efficiency, while also revealing factors that limit gains in a certain model. Our findings provide the first systematic characterization of effective reasoning with CI and demonstrate both the potential and limitations of leveraging key properties to improve CI-based reasoning.
