Search papers, labs, and topics across Lattice.
This study introduces WinDOM, a novel grounding corpus of 54,425 records generated through an automated process that extracts bounding boxes directly from the DOM of a headless Windows 11 web environment, eliminating the need for costly human annotation. The authors employ Self-Family Distillation (SFD) to enhance the performance of a small 2B parameter model by optimizing its initialization through a cold-start approach, which significantly improves out-of-distribution (OOD) performance. The results show that the SFD method outperforms traditional initialization techniques, achieving notable gains in various benchmarks, thereby demonstrating the effectiveness of small-model training strategies without relying on larger models or extensive human input.
Small models can achieve competitive performance with innovative data generation and distillation techniques, challenging the notion that bigger is always better in model training.
Small ($\sim$2B) GUI-grounding agents are attractive for on-device deployment, accessibility tooling, and low-cost iteration, but at this scale they face two open recipe questions: how to obtain bounding-box training data without expensive human annotation, and how to combine supervised fine-tuning with reinforcement learning. We address both, with the explicit goal of pushing small-model performance rather than scaling up. WinDOM is a $54{,}425$-record grounding corpus harvested by driving an open-source Windows 11 web reimplementation under headless Playwright, with bounding boxes read directly off the DOM and no OCR or human annotation. Self-Family Distillation (SFD) is a single rejection-sampling cold-start parameterised only by the teacher choice: either an EMA of the student (no external model) or a frozen larger same-family teacher. We then treat the saturation depth of the SFD cold-start as an explicit GRPO hyperparameter. On a Qwen3.5-2B student, the under-saturated cold-start is a better GRPO initialiser than the converged one: SFD-4B with Early-init RL gains $+5.4$ OOD-mean ($+3.5$ ScreenSpot-Pro, $+7.0$ OSWorld-G, $+5.8$ ScreenSpot-V2) over the base. The same-size EMA mode lands within roughly one OOD-mean point of the cross-size $4$B variant ($65.2$ vs $66.3$) without an external teacher.
