Search papers, labs, and topics across Lattice.
This paper introduces Zeva, a novel framework that enables robots to learn from their own physical interactions in real-time while keeping the policy model fixed. By utilizing a Causal Interaction Extractor and a dual-timescale causal memory, Zeva captures and retrieves relevant interaction signals to inform future actions, effectively allowing robots to evolve their capabilities during deployment. Experimental results show that Zeva outperforms existing state-of-the-art methods in both simulated and real-world manipulation tasks, demonstrating significant improvements in success rates as interaction experience accumulates.
Robots can now autonomously evolve their manipulation skills in real-time, leveraging past interactions without needing to update their underlying policy models.
Generalizable embodied manipulation remains difficult to achieve through pretraining alone, due to unseen physical conditions in the real world. We argue that robots need to learn from their own physical interactions on the fly during real-world deployment and use this knowledge to inform subsequent actions. We present Zeva, the first framework that enables in-context learning from a robot's own physical interaction experience while keeping the policy model frozen. Zeva employs a Causal Interaction Extractor to encode an executed action and its induced state change into a causal interaction signal, which is stored in a dual-timescale causal memory. For subsequent actions, relevant causal interaction signals are retrieved from memory and injected into the frozen policy model as context. Experiments in simulation and real-world manipulation demonstrate that Zeva achieves the best performance among the compared frontier VLAs and WAMs and, more importantly, enables self-evolution during deployment without gradient updates. Its success rate continues to improve as the robot accumulates interaction experience. Furthermore, the acquired interaction experience can generalize across tasks.