Search papers, labs, and topics across Lattice.
The paper introduces Search-E1, a self-evolution method for search-augmented reasoning agents that relies solely on vanilla GRPO interleaved with offline self-distillation (OFSD). By aligning the policy's inference-time distribution with its own distribution under a privileged context revealing a more efficient trajectory, Search-E1 generates dense per-step supervision without external machinery. Experiments on seven QA benchmarks show that Search-E1 achieves state-of-the-art performance among open-source models, reaching 0.440 average EM with Qwen2.5-3B.
Forget complex reward engineering and external supervision: Search-E1 shows that self-distillation alone can unlock state-of-the-art search-augmented reasoning in language models.
Post-training has become the dominant recipe for turning a language model into a competent search-augmented reasoning agent. A line of recent work pushes its performance further by adding elaborate machinery on top of this standard pipeline. These augmentations import external supervision from stronger external systems, attach auxiliary modules such as process reward models or retrospective critics, restructure the rollout itself with tree search or multi-stage curricula, or shape the reward with hand-crafted bonuses and penalties. Each addition delivers a measurable gain, but each also inflates the training pipeline and ties the recipe to resources or designs that may not always be available. We take a step back and ask whether any of this machinery is actually necessary, and propose Search-E1, a self-evolution method that lets a search-augmented agent improve through only vanilla GRPO interleaved with offline self-distillation (OFSD). After each GRPO round, the policy rolls out on its own training questions. A token-level forward KL objective then aligns the policy's inference-time distribution to its own distribution under a privileged context that exposes a more efficient sibling trajectory. Despite this simplicity, the procedure naturally provides dense per-step supervision. On seven QA benchmarks, Search-E1 reaches $0.440$ average EM with Qwen2.5-3B, surpassing all open-source baselines at both scales. Code and complete version will be made public soon.
