Search papers, labs, and topics across Lattice.
This paper introduces a novel training objective called predicted-state matching for web agents, which enhances action selection by ensuring that predicted representations can effectively distinguish true resulting states from those generated by alternative actions. By leveraging a branching web-agent dataset derived from WebArena Go-Browse trajectories, the authors demonstrate that their approach significantly outperforms traditional supervised next-state prediction methods. Key results indicate improved performance in both action ranking and end-to-end task success, showcasing the practical benefits of this discriminative training method.
Predicted-state matching enables web agents to achieve superior action selection by ensuring that their state predictions are discriminative, leading to better task performance.
Recent web agents use world models for test-time action selection by sampling candidate actions, predicting the resulting web states, and ranking them with a ranker model or a Process Reward Model (PRM). These world models are typically trained via supervised next-state prediction to generate fixed representations like HTML or AXTree snapshots. However, this objective is misaligned with the downstream ranker, which relies on predicted states being discriminative across candidates to accurately score them. To address this, we introduce predicted-state matching, a training objective where the predicted representation must distinguish the true resulting state from those reached by alternative actions. We train these models using a branching web-agent dataset derived from WebArena Go-Browse trajectories, where every decision point contains multiple alternative actions and their resulting states. Experiments on our held-out predicted-state matching benchmark show that our approach outperforms world models trained with supervised next-state prediction. We further show that our approach improves PRM-style action ranking on WebPRMBench compared with action-only PRMs and PRMs augmented with supervised-next-state world models. Finally, on WebArena-Lite, using our world model for test-time action selection improves end-to-end task success. Our project page is available at: https://dhruvpendharkar.github.io/dwm/.
BAIR