Search papers, labs, and topics across Lattice.
The paper introduces VLM-DEWM, a cognitive architecture designed to improve the robustness and verifiability of vision-language models (VLMs) for robotic planning in dynamic manufacturing environments. VLM-DEWM decouples VLM reasoning from world-state management by using a persistent, queryable Dynamic External World Model (DEWM) and structuring decisions into Externalizable Reasoning Traces (ERTs) for validation. Experiments in multi-station assembly, facility exploration, and real-robot recovery demonstrate that VLM-DEWM significantly improves state-tracking accuracy and recovery success rates compared to memory-augmented VLM baselines.
Achieve 95% recovery success in robotic manufacturing by giving vision-language models a persistent, queryable memory of the world.
Vision-language model (VLM) shows promise for high-level planning in smart manufacturing, yet their deployment in dynamic workcells faces two critical challenges: (1) stateless operation, they cannot persistently track out-of-view states, causing world-state drift; and (2) opaque reasoning, failures are difficult to diagnose, leading to costly blind retries. This paper presents VLM-DEWM, a cognitive architecture that decouples VLM reasoning from world-state management through a persistent, queryable Dynamic External World Model (DEWM). Each VLM decision is structured into an Externalizable Reasoning Trace (ERT), comprising action proposal, world belief, and causal assumption, which is validated against DEWM before execution. When failures occur, discrepancy analysis between predicted and observed states enables targeted recovery instead of global replanning. We evaluate VLM-DEWM on multi-station assembly, large-scale facility exploration, and real-robot recovery under induced failures. Compared to baseline memory-augmented VLM systems, VLM DEWM improves state-tracking accuracy from 56% to 93%, increases recovery success rate from below 5% to 95%, and significantly reduces computational overhead through structured memory. These results establish VLM-DEWM as a verifiable and resilient solution for long-horizon robotic operations in dynamic manufacturing environments.
