Search papers, labs, and topics across Lattice.
The paper introduces KbSD (Knowledge Boundary Self-Distillation), a novel framework designed to enhance decision-making in agentic search by addressing the challenges of reward sparsity in knowledge boundary calibration. By leveraging dense token-level supervision and a hint-augmented teacher model, KbSD provides explicit guidance on when to trust parametric memory versus retrieved evidence. Experimental results demonstrate that KbSD significantly improves task accuracy and reduces hallucinations, particularly in scenarios where traditional sparse rewards offer minimal guidance.
KbSD transforms the way agentic search models navigate knowledge boundaries, achieving substantial accuracy gains even in the most challenging decision-making scenarios.
Agentic search equips large language models with dynamic retrieval abilities, but existing reinforcement learning methods remain limited by reward sparsity in knowledge boundary calibration -- deciding when to trust parametric memory, when to rely on retrieved evidence, and when to abstain. Binary rewards can penalize undesirable outcomes, but provide little guidance on the reasoning process required to make calibrated decisions across different knowledge states. To address this, we propose KbSD (Knowledge boundary Self-Distillation), a framework that tackles this limitation through dense token-level supervision, outcome-level sparse rewards, and quadrant-adaptive optimization. KbSD constructs a hint-augmented teacher, architecturally identical to the student, that receives explicit knowledge boundary signals -- including parametric certainty, retrieval quality, and ground-truth answers -- to generate calibrated reasoning demonstrations. This information-asymmetric self-distillation enables dense supervision without requiring a larger external model. To further account for the heterogeneous reasoning distributions across knowledge states, we introduce a quadrant-adaptive distillation objective: reverse KL for concentrated integration, forward KL for diverse refusal, and Pareto-optimal bidirectional KL for asymmetric quadrants requiring both precision and coverage. Experiments on multiple benchmarks show that KbSD consistently improves both task accuracy and hallucination mitigation over strong baselines, with the largest gains appearing in the challenging quadrants where sparse rewards are least informative.
