Search papers, labs, and topics across Lattice.
PRAM-R, a novel autonomous driving framework, uses an LLM to dynamically select and weight sensor modalities based on environmental context and sensor diagnostics. This reduces computational cost by selectively activating sensors while maintaining performance. Experiments on nuScenes show a 6.22% modality reduction with 20% memory recall, achieving trajectory accuracy comparable to full-modality baselines.
LLMs can intelligently route sensor modalities in autonomous vehicles, slashing computation by 6% without sacrificing driving performance.
Multimodal perception enables robust autonomous driving but incurs unnecessary computational cost when all sensors remain active. This paper presents PRAM-R, a unified Perception-Reasoning-Action-Memory framework with LLM-Guided Modality Routing for adaptive autonomous driving. PRAM-R adopts an asynchronous dual-loop design: a fast reactive loop for perception and control, and a slow deliberative loop for reasoning-driven modality selection and memory updates. An LLM router selects and weights modalities using environmental context and sensor diagnostics, while a hierarchical memory module preserves temporal consistency and supports long-term adaptation. We conduct a two-stage evaluation: (1) synthetic stress tests for stability analysis and (2) real-world validation on the nuScenes dataset. Synthetic stress tests confirm 87.2% reduction in routing oscillations via hysteresis-based stabilization. Real-world validation on nuScenes shows 6.22% modality reduction with 20% memory recall while maintaining comparable trajectory accuracy to full-modality baselines in complex urban scenarios. Our work demonstrates that LLM-augmented architectures with hierarchical memory achieve efficient, adaptive multimodal perception in autonomous driving.
