Search papers, labs, and topics across Lattice.
The paper introduces Spatial Trajectory Graph (STG), a neurosymbolic paradigm that grounds LLM-based planning for Semantic-Spatial Sensor Scheduling (S3) by incorporating a "verify-before-commit" discipline. STG transforms open-ended planning into a verifiable optimization problem by grounding semantic hypotheses against a physical world model. Experiments on a campus-scale benchmark demonstrate that IoT-Brain, an STG-instantiated multi-agent system, significantly improves task success rate, reduces computational costs, and enhances reliability in real-world deployments.
LLMs can reliably control real-world sensor networks by verifying plans against a learned spatial model *before* execution, boosting success rates by 37% while slashing token usage.
Large-scale sensor networks are shifting from predefined monitoring to intent-driven operation, revealing a critical semantic-to-physical mapping gap. While Large Language Models (LLMs) excel at semantic understanding, their direct application to Semantic-Spatial Sensor Scheduling (S3) remains unreliable because plans are not grounded in physical reality. To close this gap, we introduce the Spatial Trajectory Graph (STG), a neurosymbolic paradigm built on a “verify-before-commit” discipline. STG transforms open-ended planning into a verifiable optimization problem by grounding semantic hypotheses against a physical world model and cleanly separating language inference from deterministic scheduling. We instantiate STG in our multi-agent system IoT-Brain and validate it on TopoSense-Bench, a campus-scale benchmark with 5,250 queries over 2,510 cameras. Evaluations show IoT-Brain boosts task success rate by 37.6% over the strongest search-intensive methods while running nearly 2 × faster and using 6.6 × fewer prompt tokens. Real-world deployment approaches the reliability upper bound, reducing network bandwidth 4.1 × and providing a foundational framework for LLMs to interact with the physical world with unprecedented reliability and efficiency.
