Search papers, labs, and topics across Lattice.
This paper introduces PlanSightRAG, a Visual-First Multimodal Retrieval-Augmented Generation framework designed to automate the compliance checking of civil infrastructure plans by leveraging plan imagery and advanced retrieval techniques. By integrating a multi-vector retrieval system and an agentic Planner-Retriever-Auditor-Synthesizer, the framework achieves impressive recall rates of 91.47% on zero-shot retrieval tasks and 100% verdict accuracy on synthetic compliance drawings under specific conditions. The introduction of a comprehensive benchmark dataset from five state Departments of Transportation further validates the effectiveness of PlanSightRAG in enhancing automation in compliance checking processes.
Achieving 100% verdict accuracy on compliance checks reveals the potential of visual-first approaches to revolutionize civil engineering workflows.
Civil infrastructure compliance checking has long relied on engineers manually reading legacy 2D plans; however, OCR-based automation strips away the geometry and layout essential for interpreting these plans. We present a Visual-First Multimodal Retrieval-Augmented Generation (RAG) framework called PlanSightRAG. It indexes and reasons directly over plan imagery, integrates a ColNomic-3B multi-vector retrieval, an agentic Planner-Retriever-Auditor-Synthesizer, and MaxSim heatmaps as an evidence trail. We introduce a 4,056-pair benchmark from five state Departments of Transportation (DOT) standard plans (1,898 pages). PlanSightRAG achieves 91.47% Recall@5 on zero-shot retrieval, while on a held-out Michigan DOT corpus, it achieves 91.40%. On synthetic, parametrically-generated compliance drawings, our Qwen2.5-VL-72B pipeline reaches 100% verdict accuracy only when supplied a pre-resolved rule threshold, a controlled ceiling that a non-VLM OCR baseline already reaches at 76.4%. Finally, we demonstrate autonomous visual rule-grounding by extracting numeric limits directly from a specification corpus without any human-supplied rules.
