Search papers, labs, and topics across Lattice.
The paper introduces EmbeWebAgent, a framework that embeds web agents directly into existing UIs by leveraging lightweight frontend hooks and a reusable backend workflow, enabling more robust and expressive agent behavior compared to traditional interface-level agents. This approach allows for explicit control of both frontend and backend, facilitating mixed-granularity actions and domain-specific analytics. The authors demonstrate the framework's effectiveness with minimal retrofitting effort and robust multi-step behaviors in a live UI setting.
Forget clunky screenshot-scraping web agents – EmbeWebAgent lets you embed agents directly into your UI with lightweight hooks for robust, expressive control.
Most web agents operate at the human interface level, observing screenshots or raw DOM trees without application-level access, which limits robustness and action expressiveness. In enterprise settings, however, explicit control of both the frontend and backend is available. We present EmbeWebAgent, a framework for embedding agents directly into existing UIs using lightweight frontend hooks (curated ARIA and URL-based observations, and a per-page function registry exposed via a WebSocket) and a reusable backend workflow that performs reasoning and takes actions. EmbeWebAgent is stack-agnostic (e.g., React or Angular), supports mixed-granularity actions ranging from GUI primitives to higher-level composites, and orchestrates navigation, manipulation, and domain-specific analytics via MCP tools. Our demo shows minimal retrofitting effort and robust multi-step behaviors grounded in a live UI setting. Live Demo: https://youtu.be/Cy06Ljee1JQ
