Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces ToolGraph, a novel framework that integrates schema-derived topology and history-aware controls to enhance the orchestration of multi-turn tool-using agents. By employing divergence-point preference learning, the authors construct 161 preference pairs to improve the selection process and reduce the mismatch between training and deployment. The results demonstrate a significant performance boost, with ToolGraph+DPO achieving a 16.8% increase in average reward across 375 tasks, particularly excelling in airline and retail applications.
ToolGraph+DPO not only boosts multi-turn tool-using agent performance by 16.8%, but also reveals critical insights into preference learning that could redefine agent orchestration.
Multi-turn tool-using agents must coordinate long-horizon tool sequences while tracking dialogue state and policy constraints. Existing approaches often separate inference-time orchestration from parameter-level learning, leaving tool selection weakly structured and preference updates vulnerable to train--deployment prompt mismatch. For within-benchmark self-improvement, ToolGraph combines schema-derived topology, transition weights estimated from successful rollouts, and history-aware controls for write prerequisites and repeated-search loops. We then construct 161 preference pairs by locating divergence points via state-based matching and prefix-based alignment, filtered through action-correctness annotations, and train DPO under the same ToolGraph context used at inference. Across 375 tau2-bench tasks, ToolGraph raises the weighted average reward from 0.304 to 0.338 (+11.2% relative), while ToolGraph+DPO reaches 0.355 (+16.8% over the baseline), with the DPO gain concentrated in airline and retail. Fine-grained diagnostics further show that roughly half of telecom trajectories exhaust the step budget before action execution and that chosen reward positivity is the most useful checkpoint signal across our 16 evaluated DPO configurations.