Search papers, labs, and topics across Lattice.
ViterbiPlanNet is introduced as a novel framework for procedural planning that explicitly integrates procedural knowledge via a Differentiable Viterbi Layer (DVL). The DVL embeds a Procedural Knowledge Graph (PKG) within the Viterbi decoding algorithm, enabling end-to-end optimization through smooth relaxations of non-differentiable operations. Experiments on CrossTask, COIN, and NIV datasets demonstrate state-of-the-art performance with significantly fewer parameters compared to diffusion- and LLM-based planners, highlighting improved sample efficiency and robustness.
Differentiable Viterbi decoding unlocks state-of-the-art procedural planning with an order of magnitude fewer parameters than LLM-based approaches.
Procedural planning aims to predict a sequence of actions that transforms an initial visual state into a desired goal, a fundamental ability for intelligent agents operating in complex environments. Existing approaches typically rely on large-scale models that learn procedural structures implicitly, resulting in limited sample-efficiency and high computational cost. In this work we introduce ViterbiPlanNet, a principled framework that explicitly integrates procedural knowledge into the learning process through a Differentiable Viterbi Layer (DVL). The DVL embeds a Procedural Knowledge Graph (PKG) directly with the Viterbi decoding algorithm, replacing non-differentiable operations with smooth relaxations that enable end-to-end optimization. This design allows the model to learn through graph-based decoding. Experiments on CrossTask, COIN, and NIV demonstrate that ViterbiPlanNet achieves state-of-the-art performance with an order of magnitude fewer parameters than diffusion- and LLM-based planners. Extensive ablations show that performance gains arise from our differentiable structure-aware training rather than post-hoc refinement, resulting in improved sample efficiency and robustness to shorter unseen horizons. We also address testing inconsistencies establishing a unified testing protocol with consistent splits and evaluation metrics. With this new protocol, we run experiments multiple times and report results using bootstrapping to assess statistical significance.
