Search papers, labs, and topics across Lattice.
This paper introduces GRIP, a novel framework for efficient reasoning in large language models that combines the strengths of reasoning-oriented and instruction-tuned models without requiring full model retraining. By optimizing learnable interpolation ratios for individual modules of frozen models, GRIP effectively balances accuracy and efficiency, yielding responses that are both correct and concise. Experimental results demonstrate that GRIP outperforms existing merging strategies, revealing valuable module-wise fusion patterns that enhance reasoning efficiency.
GRIP achieves superior accuracy-efficiency trade-offs in reasoning tasks by intelligently interpolating parameters from two distinct model types without retraining.
Reasoning-oriented large language models often achieve strong problem-solving performance by generating long chains of thought, but this behavior substantially increases inference cost and latency. In contrast, instruction-tuned models tend to answer more concisely, yet often lack comparable reasoning ability. This accuracy-efficiency mismatch motivates a lightweight approach that combines the strengths of both models without full model retraining. In this paper, we propose GRIP (Granular Reward-guided Interpolation of Parameters), a reward-guided parameter interpolation framework for efficient reasoning. Given a reasoning model and an instruction model with identical architectures, GRIP assigns learnable interpolation ratios to individual modules and optimizes only these ratios while keeping both source models frozen. The interpolation ratios are trained with a reward signal that favors responses that are both correct and concise. Experiments show that GRIP achieves a better accuracy-efficiency trade-off than fixed or search-based merging baselines and further reveals module-wise fusion patterns associated with efficient reasoning.
