Search papers, labs, and topics across Lattice.
This paper introduces APEx, a hierarchical framework that enhances deep research agents by organizing interaction history into trajectory memories and procedural skills, facilitating continual learning through a closed-loop architecture comprising Executor, Distiller, and Planner modules. The proposed method employs a three-stage alternating GRPO training paradigm to optimize these modules, enabling reward-guided skill distillation and skill-aligned test-time reinforcement learning. Experimental results show that APEx outperforms existing models, including GPT-5.4, by 14.7 points, demonstrating significant advancements in adaptive deep research question answering.
APEx achieves a remarkable 14.7-point improvement over GPT-5.4 by effectively distilling procedural skills for adaptive reasoning in complex question answering.
Deep research agents augment large language models with external tools to answer complex, long-horizon questions through multi-turn reasoning. Learning from prior experience is crucial for continual improvement, yet existing methods either retrieve verbose task-specific traces that burden decision-making, or distill procedural skills that remain decoupled from downstream policy adaptation. We propose APEx, a hierarchical experience utilization framework that organizes interaction history into instance-level trajectory memories and category-level procedural skills, and couples them through a closed-loop architecture of Executor, Distiller, and Planner. The three modules are optimized via a three-stage alternating GRPO training paradigm, enabling reward-guided skill distillation rather than fixed-prompt generation. At test time, distilled skills serve as procedural priors for online Planner adaptation through skill-guided test-time reinforcement learning, allowing ground-truth-free self-improvement with skill-alignment regularization to prevent policy drift. Experiments on 7 benchmarks demonstrate that APEx achieves state-of-the-art performance, surpassing GPT-5.4 by 14.7 points and the strongest memory-augmented baseline by 3.0 points.
