Search papers, labs, and topics across Lattice.
This paper introduces JITOMA, a closed-loop framework that addresses the perceptual saturation issue in long-horizon robotics by enabling just-in-time growth of 3D Scene Graphs (3DSGs). By utilizing a task heatmap to filter observations and activating only relevant memory anchors as needed, JITOMA minimizes resource-intensive operations while maintaining a global representation. Experimental results show that JITOMA significantly decreases active graph size and captioning latency, achieving stable processing times even during complex task switching.
JITOMA slashes active graph size and latency, ensuring robots can efficiently navigate long-horizon tasks without perceptual overload.
While 3D Scene Graphs (3DSGs) provide crucial structured representations for embodied agents, conventional Ahead-of-Time, build-everything-then-filter pipelines conflict with the real-time, low-latency demands of edge platforms, inducing a perceptual saturation effect via severe observation redundancy. To resolve this, we present JITOMA (Just-In-Time On-demand Memory Activation), a closed-loop framework that unifies task reasoning, perception, and memory into a just-in-time growth process. Instead of exhaustively mapping the entire environment, JITOMA leverages a top-down task heatmap at the frontend to filter continuous observations, routing minimal streams to maintain a global foundation of low-cost, dormant anchors. Upon a cognitive query, the backend Large Language Model (LLM) parses the robotic intent to dynamically awaken task-relevant anchors, triggering resource-intensive operations -- such as dense node captioning and functional inference -- exclusively within the activated local subgraph. To evaluate these dynamic capabilities and study perceptual saturation trade-offs, we introduce JITOMA-Bench, a comprehensive suite for long-horizon multi-tasking and complex multi-step reasoning. Extensive experiments demonstrate that JITOMA substantially reduces active graph size and captioning latency, while maintaining stable processing time under long-horizon task switching.
