Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces PlanSightRAG, a Visual-First Multimodal Retrieval-Augmented Generation framework designed to automate the compliance checking of civil infrastructure plans by directly indexing and reasoning over plan imagery. By integrating advanced retrieval techniques and a novel agentic architecture, PlanSightRAG achieves impressive performance metrics, including 91.47% Recall@5 in zero-shot retrieval and 100% verdict accuracy on synthetic compliance drawings under controlled conditions. The framework significantly outperforms traditional OCR methods, which only achieve 76.4% accuracy, highlighting its potential to transform compliance processes in civil engineering.
Achieving 100% verdict accuracy on compliance checks, PlanSightRAG outshines traditional OCR methods, marking a pivotal shift in automating civil infrastructure assessments.
Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.