Search papers, labs, and topics across Lattice.
This paper introduces a novel self-speculating agent that integrates tool-call prediction directly within the same model architecture, addressing the inefficiencies of existing separate speculators. By employing a joint agent-speculator reinforcement learning approach, the authors demonstrate significant improvements in predicting the next tool call, achieving Hit@1 rates of 61.2% and 66.3% for different model sizes. This unified design not only enhances performance but also maintains the agent's overall task success, showcasing a more efficient method for handling tool-call latency in language model agents.
Unifying the agent and speculator within a single model boosts next tool-call prediction accuracy by over 17% without sacrificing task performance.
Large language model agents often spend substantial wall-clock time waiting for tool call results. Tool-call speculation can hide this latency by predicting and pre-executing an agent's next tool call if the prediction matches the agent's eventual tool call, but existing speculators are typically separate draft models or cached traces that are poorly aligned with the deployed agent's own behavior. We identify this speculator-agent gap and show that the target agent itself is a strong next-call speculator. This points to a simpler design: unifying the agent and speculator within the same model. In this paper, we introduce the self-speculating agent, a single model that both solves tasks in agent mode and predicts its next tool call from partial trajectories in speculator mode, fully reusing prefix KV cache. To enable this dual-mode agent without degrading performance, we propose a joint agent-speculator reinforcement learning method, which derives speculation targets from the agent's own rollouts and alternates agent and speculator updates. Across agentic search QA and conversational tool-use agentic tasks, our method improves average next tool-call Hit@1 from 44.1 to 61.2 for Qwen3-4B and from 48.9 to 66.3 for Qwen3.5-4B, while preserving agent task success.
