Search papers, labs, and topics across Lattice.
This paper introduces NEO, a framework that leverages language-guided NeRF editing to enhance robotic manipulation through advanced scene editing techniques. It combines neural field resampling with multiview-consistent progressive inpainting for object removal and employs a teacher-student model for direct NeRF weight editing, allowing for coherent future scene state modeling. The results demonstrate significant improvements over existing methods in both visual coherence and geometric consistency during robotic manipulation tasks, supported by the introduction of the NEO-Dataset for benchmarking.
NEO achieves visually coherent and geometrically consistent scene edits that significantly enhance robotic manipulation capabilities, outperforming state-of-the-art methods.
In this paper, we present NEO, a unified framework providing language-guided NeRF editing for robotic manipulation. Our paper introduces (i) a language-guided object removal that combines neural field resampling with multiview-consistent progressive inpainting, (ii) a direct NeRF weight editing method utilizing knowledge distillation, composing original and edited NeRFs via a teacher-student model, enabling coherent modeling of future scene states before a robot executes an action, and (iii) the first benchmark (NEO-Dataset) for quantitatively evaluating NeRF scene editing methods suitable for robot manipulation. We show that our approach outperforms state-of-the-art baselines in scene editing tasks, including object removal and pick-and-place robotic experiments, yielding visually coherent and geometrically consistent edits that reduce artifacts commonly introduced by prior methods.
