Search papers, labs, and topics across Lattice.
This paper introduces the Structured Subtask Chain (SSC), a novel representation designed to improve the labeling of bimanual manipulation tasks by providing a structured format that balances readability and verification. The SSC utilizes Structured Subtask Templates (SST) to encapsulate essential action components and flexible conditions, enabling effective state-transition verification and natural language rendering. Evaluation on the BEHAVIOR-1K dataset reveals that the SSC format enhances annotation consistency and facilitates the identification of labeling anomalies across various vision-language models.
A new structured representation for manipulation tasks reveals critical labeling anomalies that traditional methods miss, enhancing both readability and verification.
Subtask labels decompose a long-horizon manipulation demonstration into shorter semantic segments for policy training and evaluation. Natural language descriptions are easy to read, but their linguistic variability makes automatic verification difficult. Rigid template formats, such as BEHAVIOR-1K's skill_annotation, are linguistically over-segmented, hindering both readability and annotation consistency. We propose the Structured Subtask Chain (SSC), a state-transition representation that bridges these extremes. A demonstration is a sequence of Structured Subtask Template (SST) entries. Each SST stores core action components (subject, predicate, object), flexible conditions (adverbial modifiers such as spatial or instrumental phrases), a base-motion field separate from arm actions, and an after-state scene graph. Built on this format, SSC supports three vision-language assisted functions: rendering SSTs as natural language, checking the assembled chain against four state-transition rules, and completing underspecified fields through a query resolution cascade. We instantiate the pipeline on BEHAVIOR-1K (50 tasks, 3 episodes per task, 2,357 annotated action cells) for logic verification and content completion, evaluating 13 selected state-of-the-art VL models as candidate verifiers and reporting labelling anomalies.
