Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces a hierarchical framework that integrates Signal Temporal Logic (STL) with Vision-Language-Action (VLA) models to enhance the interpretability and execution of complex language instructions in robotic systems. By decomposing high-level language commands into subtasks and generating precise STL specifications, the framework allows for effective low-level policy selection and execution, ensuring that spatial, temporal, and logical requirements are met. Evaluated in a real-world tabletop domain, the approach significantly improves the precision, reliability, and interpretability of robot planning, showcasing the utility of formal specifications in real-time execution contexts.
Formalizing language-derived robot commands with STL transforms vague instructions into precise, executable plans, enhancing reliability in complex tasks.
Vision-language-action (VLA) models have shown impressive generalization, but often lack interpretability and can struggle to follow precise natural language instructions that encode spatial, temporal, and logical requirements. We propose a hierarchical framework that uses Signal Temporal Logic (STL) as a shared representation connecting high-level language understanding with low-level robot execution. A high-level policy leverages a VLM to decompose language instructions into high-level subtasks, generate STL specifications for each subtask, and choose a low-level policy for executing each subtask. The STL specifications translate language-derived intent into precise constraints, and the low-level policy selection determines whether those constraints are enforced directly through STL-guided model-predictive control or monitored during execution of a learned policy for perceptually complex, or contact-rich behaviors. By integrating STL into plan validation, low-level policy, subtask monitoring, and replanning, our framework enables language-derived plans to be checked, optimized, and revised at runtime using a common formal structure. We evaluate the approach on a real-world tabletop domain, demonstrating how formal specifications can improve the precision, reliability, and interpretability of language-conditioned robot planning.