Search papers, labs, and topics across Lattice.
This paper investigates the limits of scaling ego-centric human video for training world models, finding that while agent modeling fidelity improves significantly with 30,000 hours of data, object interaction fidelity lags considerably and saturates slowly. The authors demonstrate that agent gains are not solely data-dependent, and a refined visual conditioning design can achieve high agent fidelity with less data, allowing for isolated object fidelity measurement. A novel supervision scheme reallocates model capacity towards object dynamics, yielding improvements but still leaving a substantial gap, suggesting that future progress hinges on training methodology rather than just data volume.
Ego-centric video scaling saturates agent modeling long before object interaction fidelity, revealing a fundamental bottleneck in current world model training paradigms.
World models offer a promising alternative to physics-based simulators, yet remain far from practical deployment. We ask how far scaling ego-centric human video takes them, using a dataset of 30,000 hours spanning over 1,000 scene types and 14,000 contributors. Rather than relying on opaque downstream metrics, we directly evaluate agent and object-interaction fidelity on a challenging out-of-distribution benchmark. Increasing training data by 100x improves both, but unevenly: the agent is modeled well, while object fidelity remains far lower and improves slowly. We show that the agent gains need not come from data, and a careful visual conditioning design saturates fidelity with a fraction of it, which lets us measure object fidelity on its own and discover its saturation point. We then introduce a supervision scheme that shifts capacity from scene appearance toward object dynamics, improving object fidelity though a substantial gap remains. Finally, our conclusions transfer to downstream humanoid modeling. Overall, our results suggest that scaling ego-centric data brings agent modeling close to its limit while leaving its effects on the world far behind, and that closing this gap will depend on how models are trained, not only on how much data they see.