Track synthesis, materials and catalysis as it is published.
Search papers, labs, and topics across Lattice.
This paper investigates object memory in learned world models under partial observability by comparing a grid-based representation (FloWM) with a node-based representation using an oracle-fed Kalman tracker. While FloWM's grid-based approach excels when its internal probe is used and benefits from prediction write-back, it struggles with out-of-view objects and novel velocities compared to the more memory-efficient node-based tracker. The research highlights trade-offs in representation choice, memory footprint, and adaptability to dynamic environments.
A grid-based world model can achieve 89% out-of-view object recall and outperform a Kalman tracker in view, but a node-based approach offers drastically superior memory efficiency and robustness to unmodeled velocities.
A robot's world model must remember objects that leave the camera's view. FloWM, a learned world model, keeps them in a spatial grid of features that it moves with five built-in velocities. A node memory instead keeps each object's position and velocity as numbers. We compare them in FloWM's own block world, with an oracle-fed Kalman tracker as the node memory. How the grid is read changes its score: a single-cell probe finds 66% of out-of-view blocks, while FloWM's own probe 89%. FloWM also scores 8 points higher in view although no frame is shown, which we attribute to its write-back of its own predictions. Out of view, the tracker reaches FloWM's range in 14KB against 5.6MB, with no probe. On diagonal motion, outside its five velocities, FloWM finds 11% of blocks even after fine-tuning, while the unchanged tracker keeps 93%. Faster blocks defeat both until one more allowed speed takes the tracker to 94%, where FloWM's built-in shifts would need 25 copies of its grid. A learned node memory must still replace the oracles and handle velocity under noise, long gaps, and the bounce.