Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of negation-constrained dexterous grasping in language-driven models, which typically struggle when instructed to avoid certain regions. The authors introduce an innovative inference-time framework that utilizes Sequential Monte Carlo and classifier-free guidance to effectively navigate grasping instructions without requiring specific training examples for negation. Their method significantly reduces the violation rate of forbidden regions from 57.9% to 17.2% on the newly constructed NegGrasp benchmark, while also enhancing overall task success rates.
Negation-constrained grasping can be achieved without any specific training examples, dramatically lowering error rates in forbidden region violations.
Language-driven dexterous grasp models, such as DextER, perform well when instructions specify where to grasp, but we find they fail systematically when an instruction also specifies where not to grasp (e.g.,"grasp the handle but avoid the body"). Existing training corpora, DexGYSNet among them, contain virtually no avoidance instructions, and collecting examples for every possible constraint is impractical. Moreover, because every part mentioned during training denotes a contact target, models may interpret a forbidden part as another region to grasp rather than one to avoid. We therefore introduce an inference-time framework for negation-constrained dexterous grasping that requires no negation-specific training examples. Combining Sequential Monte Carlo with classifier-free guidance, our method guides sampling toward the instructed part while pruning candidates headed for the forbidden region, without any negation examples during training. A frozen 3D part-grounding model localizes the forbidden region from the language instruction. To evaluate this setting, we construct NegGrasp, a benchmark of paired positive/negative instructions with constraint-aware metrics that credit a grasp only if it both accomplishes the task and respects the stated constraint. On NegGrasp, our method reduces the violation rate of the strongest baseline from 57.9% to 17.2% while improving both constraint-aware and physical success.