Track new results across pure and applied mathematics.
Search papers, labs, and topics across Lattice.
W2Rep is a self-supervised masked feature-prediction framework designed to supervise an independently encoded single-image encoder using cross-frame temporal evolution in video. By conditioning a latent predictor on visible multi-frame video context, spatial query locations, and signed temporal displacement ($\Delta t$), the objective simultaneously forces the image encoder to retain temporally stable features and the video path to supply missing dynamic context. Evaluated across model scales and downstream recognition tasks, W2Rep significantly outperforms standard baselines on both frozen and fine-tuned benchmarks, operating effectively at both static image and joint-video granularities.
Watching physical scenes change over time turns out to be an ideal supervisory signal for static visual features, allowing an independently encoded frame supervised across signed temporal intervals to excel at both single-image and multi-frame downstream tasks.
Images capture the world at one moment, whereas video reveals how it changes. Image self-supervision learns spatial structure from a single moment, while video methods commonly learn temporal relationships inside a representation computed jointly from several frames. We ask whether watching a scene change can instead improve features available from one image without sacrificing the ability to represent video. We introduce W2Rep, a masked feature-prediction framework in which an independently encoded source image participates in prediction at the same or another moment. The predictor is conditioned on visible video context, the queried location, and the signed time interval between source and target. This gives the cross-frame objective two complementary roles: the image path learns features that remain useful across time, while the video path must gather evidence that is missing from the source image. Across model scales and downstream tasks, W2Rep improves frozen and fine-tuned recognition under our comparison protocol, while joint video encoding provides further gains over frame-wise aggregation. Controlled experiments show that these gains depend on directly updating the source-image features and on using both video context and temporal displacement. Overall, change across a video can supervise a visual encoder whose representations remain useful at either image or video granularity. Code is available at~\href{https://wenooi.github.io/W2Rep}{https://wenooi.github.io/W2Rep}.