Search papers, labs, and topics across Lattice.
CMU ML
Tsinghua AIBeihangHKUHKUSTImperialKCLManchesterSEUStevensTuojing IntelligenceZJUJul 5, 2026arXiv:2607.04234This paper introduces SoftVTBench, a novel benchmark designed to evaluate the safety of robotic manipulation of deformable objects, addressing the gap in existing benchmarks that focus solely on task completion. By incorporating multi-modal sensory inputs and defining metrics for both Goal Success and Safety Success, the authors reveal that many policies achieving goal completion still compromise physical safety. Experiments demonstrate that integrating tactile sensing significantly enhances safety outcomes without sacrificing overall task success, highlighting the importance of safety in robotic manipulation tasks.
Many robotic policies that seem successful in manipulation tasks actually compromise safety, with SoftVTBench revealing a stark contrast between goal completion and physical safety metrics.
Deformable object manipulation poses challenges beyond task completion: successful execution must also maintain safe physical interaction, holding the object stably without slip or drop while avoiding excessive deformation. However, existing manipulation benchmarks are predominantly success-oriented and rarely evaluate whether a policy remains physically safe throughout execution. We present SoftVTBench, a safety-aware visuo-tactile benchmark for physically constrained deformable object manipulation. Built in Isaac Sim with finite-element-simulated deformable objects, SoftVTBench provides multi-view RGB observations, RGB tactile sensing with marker motion, proprioception, and language instructions, and defines four matched task suites over object type (deformable vs. rigid) and variation axis (object vs. spatial). It separately reports Goal Success and Safety Success; the latter additionally requires no drop and peak deformation below a calibrated object-specific threshold, measured from policy-hidden privileged Finite Element Method (FEM) states. We implement pi0.5-based baselines under this protocol. Experiments show that success-only evaluation substantially overstates policy performance, as a large fraction of goal-completing rollouts still violate physical safety. Furthermore, incorporating tactile sensing improves Safety Success (e.g., from 21.4% to 35.6% on object-centric deformable tasks) and reduces object deformation during execution, while maintaining comparable Goal Success. SoftVTBench provides a reproducible benchmark for studying visuo-tactile deformable manipulation under physical interaction constraints.
