Search papers, labs, and topics across Lattice.
SpaceDex is a hierarchical framework for dexterous manipulation in tiered workspaces, addressing challenges like occlusion and height constraints. It uses a Vision-Language Model (VLM) planner for high-level spatial reasoning and an arm-hand Feature Separation Network to decouple arm trajectory control from grasp mode selection. Experiments on 100 real-world trials with unseen objects show SpaceDex achieves a 63.0% success rate, significantly outperforming a tabletop baseline (39.0%).
Hierarchical planning with vision-language models and decoupled arm-hand control unlocks dexterous grasping in cluttered, tiered workspaces where traditional methods falter.
Generalizable grasping with high-degree-of-freedom (DoF) dexterous hands remains challenging in tiered workspaces, where occlusion, narrow clearances, and height-dependent constraints are substantially stronger than in open tabletop scenes. Most existing methods are evaluated in relatively unoccluded settings and typically do not explicitly model the distinct control requirements of arm navigation and hand articulation under spatial constraints. We present SpaceDex, a hierarchical framework for dexterous manipulation in constrained 3D environments. At the high level, a Vision-Language Model (VLM) planner parses user intent, reasons about occlusion and height relations across multiple camera views, and generates target bounding boxes for zero-shot segmentation and mask tracking. This stage provides structured spatial guidance for downstream control instead of relying on single-view target selection. At the low level, we introduce an arm-hand Feature Separation Network that decouples global trajectory control for the arm from geometry-aware grasp mode selection for the hand, reducing feature interference between reaching and grasping objectives. The controller further integrates multi-view perception, fingertip tactile sensing, and a small set of recovery demonstrations to improve robustness to partial observability and off-nominal contacts. In 100 real-world trials involving over 30 unseen objects across four categories, SpaceDex achieves a 63.0\% success rate, compared with 39.0\% for a strong tabletop baseline. These results indicate that combining hierarchical spatial planning with arm-hand representation decoupling improves dexterous grasping performance in spatially constrained environments.
