Search papers, labs, and topics across Lattice.
This paper introduces ISE, a novel three-stage synthesis paradigm designed to create a dataset for training operating system agents that captures structured user intents, multi-turn task delegation, and grounded tool execution. By generating approximately 44,000 unique intents and simulating realistic user interactions, the authors produced over 23,000 multi-turn trajectories that reflect authentic execution outcomes. Fine-tuning on this dataset significantly improved the performance of the Qwen3-8B model on agent tool-use tasks, achieving a pass rate of 37.7%, surpassing both zero-shot GPT-4o and the larger Qwen3-32B model.
Authentic execution dynamics in multi-turn interactions can double the performance of OS agents, challenging the efficacy of larger models.
Training capable OS agents requires data that simultaneously captures structured user intents, multi-turn task delegation, and grounded tool execution--properties absent from existing datasets. We propose ISE (Intent -> Simulate -> Execute), a three-stage synthesis paradigm that addresses these gaps jointly. Stage 1 constructs roughly 50000 structured intents via a 4D framework (Persona x Domain x Task x Complexity); after deduplication the pool contains 43956 unique intents and attains a Vendi Score of 61.57 over the entire pool on mpnet-base-v2 embeddings (cosine kernel, q=1). Stage 2 drives multi-turn user-agent interaction through a role-locked user simulator that grounds each user turn in actual execution outcomes, producing 23132 complete trajectories averaging 8.12 user turns and 68.24 total dialogue turns. Stage 3 runs every tool call inside a live, isolated OS workspace, generating authentic failure-recovery dynamics instead of simulated responses. Fine-tuning on ISETrace improves ClawEval pass@1 from 19.3 to 37.7 using Qwen3-8B on agent tool-use tasks with a standard protocol. This result outperforms zero-shot GPT-4o and the larger Qwen3-32B base model which is four times bigger. An ablation on Stage 2 proves multi-turn simulation brings a large portion of the performance gain. We release all source code and dataset at https://github.com/Valiere01/ISE-Trace.
