Search papers, labs, and topics across Lattice.
This paper introduces NarrativeFlow, a flow-based vision-language-action model that represents robot motion as continuous velocity fields, enabling embodiment-agnostic data utilization for language-conditioned manipulation. By employing a flow-matching formulation conditioned on language, NarrativeFlow generates physically consistent robot flows, addressing the limitations of existing methods that rely on sparse approximations or lack language conditioning. Experiments demonstrate NarrativeFlow's superior performance over baselines on both simulated and real-world manipulation tasks.
Robot velocity fields, not just sparse trajectories, unlock embodiment-agnostic learning for language-conditioned manipulation.
We focus on language-conditioned flow-based manipulation, where robot flows (robot velocity fields) serve as embodiment-agnostic, motion-centric representations for leveraging data collected from multiple robot platforms. This task is crucial because language-conditioned manipulation is essential for practical robotic systems, yet scaling robot foundation models remains limited by the labor-intensive collection of embodiment-specific data. Existing methods either coarsely approximate robot flows with sparse keypoint displacements, or cannot handle language-conditioned manipulation. To address this limitation, we propose NarrativeFlow, which models robot flows as continuous velocity fields using a flow-matching formulation conditioned on language. Accordingly, NarrativeFlow generates robot flows that are physically consistent with real-world manipulation. To validate NarrativeFlow, we have conducted experiments on standard datasets for language-conditioned manipulation. The experimental results show that NarrativeFlow outperforms representative baseline methods on standard evaluation metrics. Furthermore, through real-world experiments, we show that NarrativeFlow achieves higher success rates than baseline methods across multiple manipulation tasks. The project page is available at https://shota0520.github.io/NarrativeFlow-project-page/