Search papers, labs, and topics across Lattice.
This paper introduces DF$^3$, a novel framework for world modeling in autonomous navigation that eliminates the need for decoders by operating entirely within the latent space. By leveraging learnable spatial queries and a Motion-Aware Context Fusion mechanism, DF$^3$ efficiently forecasts future state representations directly from historical data. Experimental results show that DF$^3$ achieves competitive performance against state-of-the-art methods while significantly improving computational efficiency and flexibility for integrated perception and control tasks.
Eliminating decoders in world modeling could revolutionize efficiency in autonomous navigation systems, achieving state-of-the-art performance with less computational overhead.
Forecasting future states from video sequences is a critical challenge for autonomous robotic systems and a fundamental objective of world modeling. Prior generative methods operating at the pixel level inevitably overemphasize task-irrelevant details, leading to prohibitive computational overhead. While latent-based approaches attempt to mitigate this by predicting features directly, the persistent reliance on heavy decoders for state-to-task mapping remains a computational bottleneck. In this work, we propose Decoder-Free Feature Forecasting (DF$^3$), a novel framework that models world evolution entirely within the latent space and directly derives task outputs, completely eliminating the need for a decoder. Specifically, DF$^3$ injects learnable spatial queries into the terminal blocks of a frozen vision foundation model to extract future state representations directly. By employing a lightweight, unified Motion-Aware Context Fusion (MACF) mechanism that seamlessly integrates coarse flow warping with fine-grained latent cross-correlation, these queries interact with historical token representations to explicitly align and forecast the feature of the next frame. Subsequently, a specialized set of task queries probes these forecasted features for the downstream task. Extensive experiments on public benchmarks and zero-shot deployment in a robotic simulator demonstrate that DF$^3$ achieves performance comparable to state-of-the-art methods while offering superior efficiency and flexibility for integrated perception and control.
