Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The paper introduces $\mathcal{N}_0$-Foundation, a comprehensive framework for tactile-enabled embodied manipulation that integrates advanced tactile sensing hardware with large-scale multimodal data collection. The authors developed a robust infrastructure for data gathering, resulting in NeoData, a dataset comprising over 30,000 hours of synchronized visual and tactile demonstrations across various robotic embodiments and tasks. Key findings reveal that policies trained on tactile representations benefit more from the physical contact state than from the specific appearance of tactile signals, underscoring the importance of tactile feedback in manipulation tasks.
Tactile representations can significantly enhance robotic manipulation by focusing on physical contact rather than device-specific tactile appearances.
We present $\mathcal{N}_0$-Foundation, a paradigm for tactile-enabled embodied manipulation, which integrates tactile sensing hardware, large-scale multimodal data, tactile representation learning, and standardized evaluation. First, we engineer the infrastructure for scalable data collection, including a vision-based tactile sensor, a tactile Universal Manipulation Interface (UMI), and a synchronized visuo-tactile data collection system supporting both robot embodiments and UMI-based demonstrations. Leveraging this infrastructure, we construct NeoData, which contains more than 30000 hours of synchronized visual and tactile demonstrations, spanning six embodiments, 450 tasks, and billions of paired RGB and tactile frames collected through a mixture of real-robot teleoperation and UMI-based demonstrations. To facilitate open research, we further release OpenNeoData, a 5000-hour open-source subset of NeoData. The dataset addresses a central limitation of existing manipulation corpora, critical for deformable-object manipulation, precise assembly, delicate force control, and sustained surface interaction. Capitalizing on the large-scale, heterogeneous tactile measurements, we propose NeoForce, a visuo-tactile representation model that learn transferable tactile representations across different sensor designs. To enable systematic evaluation of tactile embodied models built upon our infrastructure, datasets and tactile representations, we further propose a comprehensive benchmark, which combines the real-world NeoReal suite and the simulated NeoSim suite for standardized evaluation. Experiments across both suites show that policies benefit from the physical contact state rather than from the device-specific appearance of the tactile signal. We release the dataset, the representation, and the benchmark, aiming at supporting future work on tactile-enabled embodied manipulation.