Search papers, labs, and topics across Lattice.
This paper introduces FCPAgent, a falsifiable commitment planning framework designed to enhance the reliability of long-horizon web agents by incorporating a structured approach to evidence evaluation. Each plan step is represented as a Falsifiable Commitment Unit (FCU), which includes confirming and falsifying evidence along with a confidence score, enabling agents to dynamically assess the validity of their actions. The implementation of a plan-test-repair loop, combined with lightweight evidence matching and LLM-based verification, results in a 13.8% relative improvement in success rates on long-horizon tasks compared to existing methods.
FCPAgent redefines how web agents validate their actions, achieving a 13.8% boost in success rates on complex tasks by integrating falsifiable commitments into planning.
Long-horizon web agents often go off track before final failure: a trajectory can remain locally plausible even after the current state, reused skill, or plan assumption no longer supports the user instruction. Existing agents can plan, reflect, or reuse experience, but their plans rarely specify the evidence under which an active step should still be trusted. We propose FCPAgent, a falsifiable commitment planning framework for robust long-horizon web agents. FCPAgent represents each plan step as a Falsifiable Commitment Unit (FCU): a subgoal grounded in a reusable skill, together with confirming evidence, falsifying evidence, and a confidence score. Execution is organized as a plan-test-repair loop. The hybrid commitment testing module checks candidate actions before they modify the browser and checks observations after execution; for efficiency, it combines lightweight evidence matching with LLM-based diagnostic verification. When evidence falsifies a commitment, scope-aware repair localizes the contradiction to the execution, skill, or planning level and revises the smallest adequate part. On WebArena, FCPAgent achieves a 13.8% relative improvement in average success over the strongest baseline, with especially large gains on long-horizon tasks.
Tsinghua AI