Search papers, labs, and topics across Lattice.
This paper introduces AdaTurn, a budget-aware framework designed to enhance the performance of active visual perception agents by conditioning them on the allowed number of turns during deployment. By addressing the issue of catastrophic truncation, where agents fail to produce valid answers due to fixed rollout budgets, AdaTurn employs a novel component called Forced-Answer DAPO (FA-DAPO) to convert over-budget scenarios into trainable decision-making steps. The results show significant improvements in low-budget accuracy, exemplified by an increase from 36.7% to 47.6% on the VisualProbe-Medium task, while maintaining performance across varying budget scenarios and multimodal benchmarks.
Budget-aware training can boost low-turn performance by over 10% while maintaining scalability across diverse tasks.
Active visual agents solve fine-grained image tasks by interleaving reasoning with image-grounding actions across multiple turns. However, deployment-time rollout budgets are rarely fixed: some requests permit long rollouts, while others require the agent to act under a tight turn limit. Existing methods train the policy as if the rollout budget were hidden, so when the available budget is smaller than the trajectory the agent prefers, the interaction is often truncated before any valid answer is produced; we term this failure \emph{catastrophic truncation}. To overcome this challenge, we present AdaTurn, a budget-aware framework that conditions the agent on the allowed number of turns and explicitly trains the boundary behavior induced by the budget. Our key component, Forced-Answer DAPO (FA-DAPO), converts the over-budget event from a masked or penalized failure into a trainable final-decision step, teaching the model to synthesize partial evidence when further tool use is no longer possible. We further randomize rollout budgets during both training and inference and introduce a load-balanced scheduler that makes such operations practical. AdaTurn substantially improves low-budget accuracy, for example raising VisualProbe-Medium from 36.7% to 47.6% at four turns, while preserving strong scaling at larger budgets and transferring effectively to multiple backbones and general multimodal benchmarks.
