Search papers, labs, and topics across Lattice.
This paper introduces ATOM-Bench, a benchmark designed to evaluate atomic skills and compositional generalization in robotic manipulation policies. By factorizing tasks into motor and instruction atoms, it assesses 30 atomic tasks and 24 compositional tasks, revealing that while policies can learn basic instruction-grounding skills, they often falter on fine-grained motor tasks and struggle to transfer these skills to new task structures. The findings highlight a critical gap in current manipulation policies, where strong performance in atomic skills does not guarantee success in more complex compositional scenarios.
Current manipulation policies may excel at basic tasks but often fail to transfer learned skills to new contexts, exposing a significant gap in their real-world applicability.
Generalist manipulation policies are increasingly presented as foundation models for robotic control, but their real-world generalization remains difficult to diagnose. A policy may succeed on demonstrated tasks while still failing to execute fine-grained atomic skills or recombine learned skills in new task structures. We introduce \textbf{ATOM-Bench}, a real-world benchmark for evaluating both atomic skills and compositional generalization in manipulation policies. ATOM-Bench factorizes tabletop manipulation into motor atoms and instruction atoms, and contains 30 atomic tasks and 24 held-out compositional tasks across paired single-arm and dual-arm robot tracks. We collect 3,000 human demonstrations for atomic fine-tuning and release both the demonstration data and evaluation rollout data to support reproducible real-world evaluation. Policies are fine-tuned on atomic tasks and evaluated on both atomic skill acquisition and held-out compositional tasks. We further introduce Atomic Score (AS) and Compositional Failure Share (CFS) to distinguish failures caused by weak atomic skills from failures caused by limited compositional reuse. Through 2,700 physical rollouts on five representative manipulation policies, we find that current policies can acquire simple instruction-grounding skills, but still struggle with fine-grained motor atoms, counting, and logical filtering. More importantly, strong atomic performance does not reliably transfer to held-out compositional tasks. ATOM-Bench provides a diagnostic testbed for studying whether failures arise from weak motor execution, poor instruction grounding, or limited compositional reuse.
