Search papers, labs, and topics across Lattice.
This paper introduces a reinforcement learning fine-tuning framework for Computer-Use Agents (CUAs) that leverages autonomous vision-language evaluation to provide scalable supervision signals in complex graphical user interfaces. By modeling the feedback from imperfect evaluators as a noisy binary reward channel and employing a noise-corrected reward estimator during Proximal Policy Optimization, the authors significantly enhance task success rates. Experiments demonstrate that this approach yields an average improvement of 12.6 percentage points over zero-shot performance and 5.1 points over raw evaluator fine-tuning, highlighting the efficacy of autonomous evaluation in RL applications for CUAs.
Autonomous evaluation can boost task success rates for GUI agents by over 12% when noise is effectively modeled and corrected.
Computer-Use Agents (CUAs) execute high-level user goals by perceiving and acting directly within graphical user interfaces. However, reinforcement learning for CUAs remains difficult because open-ended desktop environments rarely provide scalable, machine-readable reward signals: task success is often visually grounded and hard to specify with handcrafted reward functions or dense manual labels. We propose an RL fine-tuning framework that uses autonomous vision-language evaluation as a scalable supervision signal for GUI agents. Given a final screenshot and the original instruction, a Vision-Language Model judges task completion and provides terminal feedback without task-specific heuristics or manual labels during policy optimization. Because autonomous evaluators are imperfect, we model their feedback as a noisy binary reward channel and derive a noise-corrected reward estimator for Proximal Policy Optimization. Experiments across macOSWorld, Windows Agent Arena, and OSWorld show that corrected evaluator rewards outperform both zero-shot baselines and raw evaluator rewards, improving success rates by an average of 12.6 percentage points over zero-shot performance and 5.1 points over raw evaluator fine-tuning. These results suggest that autonomous evaluation can serve as a practical reward signal for RL in GUI environments when evaluator noise is explicitly modeled and corrected.
