Search papers, labs, and topics across Lattice.
This paper introduces SkillMigrator, an innovative agent that enhances the efficiency of large language model (LLM) web agents by learning and reusing transferable interaction patterns (TIPs) based on layout structure rather than specific element references. By enabling the retrieval of these TIPs during execution, SkillMigrator significantly reduces the number of LLM actions required for successful web interactions, achieving an 8-10% reduction in action count on benchmarks like WebArena and Mind2Web. This advancement addresses the limitations of traditional skill libraries, which often rely on instruction similarity and coarse metadata, thereby improving both latency and cost-effectiveness in LLM deployments.
SkillMigrator slashes LLM action counts by up to 10% by leveraging layout structure for skill reuse, transforming web agent efficiency.
Large language model (LLM) web agents are usually deployed as tool callers: each turn, the model reads a fresh page observation and emits one structured tool action. When every action is a low-level primitive, horizons grow quickly and so do policy-facing LLM completions, dominating latency and cost on benchmarks such as Mind2Web and WebArena. Recent systems therefore wrap repeated interaction fragments as web skills: callable tools built from successful trajectories or induced programs, so one call can replace several primitives. However, prior skill libraries are still triggered mainly by instruction similarity or coarse site metadata, which yields low skill reuse on held-out sites and leaves much of the potential step and token reduction on the table. We present SkillMigrator, an agent that learns reusable web skills and transfers them across sites by matching layout structure rather than specific element references. Each induced skill is stored as a transferable interaction pattern (TIP): the skill paired with a structural sketch of the snapshot at induction time. At test time, SkillMigrator retrieves TIPs by layout similarity and grounds their references on the live page. The rest of the stack is standard: accessibility-snapshot observations with stable references, and fixed tool calling over primitives plus skill invocations. Compared with the state-of-the-art approaches, SkillMigrator reduces the average LLM-action count on successful trajectories by 8-10% across both WebArena and Mind2Web at matched success rate.
DAMO