Search papers, labs, and topics across Lattice.
This study investigates the semantic handoff failures that occur when robots execute long-horizon tasks by composing multiple language-conditioned skills. Using an agent-orchestrated vision-language-action execution harness, the authors evaluate skill performance under both ideal and chained initial states, revealing that while individual skills can achieve high success rates, the transition between skills often leads to task stalls due to issues like next-skill readiness and target grounding. The findings highlight a critical gap between isolated skill competence and the robustness required for reliable long-horizon task completion, suggesting that future skill libraries must address the complexities of real-world execution scenarios.
Despite high success rates for isolated skills, robots frequently stall in long-horizon tasks due to semantic handoff failures, revealing a hidden challenge in skill composition.
Long-horizon household tasks require robots to compose many language-conditioned skills, yet the boundary between consecutive skills is rarely explicit. A skill may satisfy its own postcondition while leaving the robot, objects, or camera views in a state from which the next skill cannot reliably start. We study this semantic handoff problem in BEHAVIOR-1K through an agent-orchestrated vision-language-action execution harness. The harness invokes $\pi_{0.5}$-based skill checkpoints trained from cleaned BEHAVIOR-1K demonstrations, assigns each skill typed arguments and a step budget, and uses multi-view vision-language model verification to decide whether execution should advance, retry, or replan. To separate isolated skill competence from long-horizon compositional robustness, we evaluate the same checkpoints under two initial-state distributions: clean skill-boundary snapshots and chained terminal states produced by previous skills. Selected navigation, grasping, placement, and door-opening skills achieve 77--100% success from clean snapshots under human-reviewed verification, yet composed rollouts still frequently stall from chained states. Execution traces attribute these failures to next-skill readiness, target grounding, and low-level control execution, revealing a substantial gap between single-skill success and reliable long-horizon task completion. These findings turn near-zero end-to-end task success into actionable diagnostics, showing that future VLA skill libraries must learn robustness to the messy chained-state distribution that clean demonstrations systematically underrepresent.
