Search papers, labs, and topics across Lattice.
This paper introduces Tool-Internalized Reasoning (TInR), a novel approach that internalizes tool knowledge into LLMs to improve reasoning without relying on external tool documentation. They propose TInR-U, a framework trained via a three-phase pipeline involving bidirectional knowledge alignment for tool internalization, supervised fine-tuning, and reinforcement learning with TInR-specific rewards. Experiments demonstrate that TInR-U outperforms existing methods in both in-domain and out-of-domain settings, showcasing its effectiveness and efficiency.
LLMs can reason more effectively and efficiently by internalizing tool knowledge, eliminating the need for external documentation and reducing inference costs.
Tool-Integrated Reasoning (TIR) has emerged as a promising direction by extending Large Language Models' (LLMs) capabilities with external tools during reasoning. Existing TIR methods typically rely on external tool documentation during reasoning. However, this leads to tool mastery difficulty, tool size constraints, and inference inefficiency. To mitigate these issues, we explore Tool-Internalized Reasoning (TInR), aiming at facilitating reasoning with tool knowledge internalized into LLMs. Achieving this goal presents notable requirements, including tool internalization and tool-reasoning coordination. To address them, we propose TInR-U, a tool-internalized reasoning framework for unified reasoning and tool usage. TInR-U is trained through a three-phase pipeline: 1) tool internalization with a bidirectional knowledge alignment strategy; 2) supervised fine-tuning warm-up using high-quality reasoning annotations, and 3) reinforcement learning with TInR-specific rewards. We comprehensively evaluate our method across in-domain and out-of-domain settings. Experiment results show that TInR-U achieves superior performance in both settings, highlighting its effectiveness and efficiency.
