Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To solve the tendency of video models to double-count re-observed entities in continuous streams, this work formulates StaMina, a framework that learns state-conditioned updates over recurrent visual context to maintain object visibility, persistent identities, and completed-event histories. The authors train an object and event recurrence end-to-end using a differentiable state transition loss constrained by trajectory endpoints alongside visibility and association objectives over 39.8K curated queries. Evaluated on the causal SVCBench benchmark across prefix replay and persistent streaming settings, an 8B model achieves 44.9 and 38.2 Gaussian Precision Accuracy, outperforming standard Counting-SFT by up to 10.9 points.
Streaming video models consistently fail at continuous counting because they cannot distinguish re-observed entities from new instances—a failure mode resolved by framing online counting as differentiable state-machine updates over persistent visual trajectories.
Continuous video counting requires distinguishing new observations from new objects or completed events. We introduce StaMina (State Maintenance), which learns to maintain counting state through state-conditioned updates. Recurrent visual context supports recognition; learned transitions maintain visibility, persistent identities, and completed-event records. A differentiable recurrence trains event transitions over legal paths constrained by count endpoints; visibility and association objectives train the object branch. A multi-source pipeline organizes 39.8K spatial queries and complementary event annotations into counting trajectories. On SVCBench, we evaluate counting adaptation with partial video overlap and held-out groups of linked annotations. Under prefix replay (Full) and persistent streaming (Stream), 4B and 8B models reach 41.9/36.4 and 44.9/38.2 Gaussian Precision Accuracy, respectively. The 8B model gains 10.9/3.2 points over Counting-SFT on the same queries. Matched-graph comparisons isolate phase conditioning and trajectory supervision, assessing training objectives alongside hard decisions. Online video benchmarks and count-conditioned decisions assess online understanding and task eligibility. Project Page: https://PLACEHOLDER.github.io/StaMina/