Search papers, labs, and topics across Lattice.
StreamForce is a novel streaming video generation framework that integrates physically grounded control through continuous force inputs, addressing limitations of prior models that either separate force types or rely on non-causal processing. The framework employs a unified force representation as a control signal, enabling real-time responsiveness to both local and global, time-varying forces while maintaining stable photometric and dynamic realism. Achieving up to 16.6 FPS on a single GPU, StreamForce sets a new benchmark in force adherence and motion realism in video generation tasks.
StreamForce achieves real-time video generation with physical control, outperforming traditional models in both responsiveness and realism.
We introduce StreamForce, a streaming video generation framework that enables physically grounded control through continuous force inputs. Unlike prior video models that train separate models for different force types, assume fixed forces, or rely on non-causal processing, StreamForce is a causal and unified model that responds instantly and coherently to both local and global, time-varying forces. To achieve this, we design a unified force representation as a control signal and develop a distillation pipeline for force-controllable video generation. Our model combines autoregressive efficiency with force responsiveness, sustaining stable photometric and dynamic realism. StreamForce runs at up to 16.6 FPS on a single GPU, achieving state-of-the-art performance in both force adherence and motion realism. Project website: https://neu-vi.github.io/StreamForce/
BAIR