Search papers, labs, and topics across Lattice.
This paper introduces MechVQA, the first comprehensive dataset for evaluating Multimodal Large Language Models (MLLMs) on mechanical drawing understanding, consisting of 3.3k images and 21K question-answer pairs across various tasks. The authors develop the MechVL model using a multi-stage training approach, which significantly improves performance on this challenging domain, outperforming the best closed-source baseline by 7.57 percentage points. This work addresses the critical gap in MLLMs' ability to interpret complex mechanical drawings, which is essential for applications in mechanical design and inspection.
MLLMs can now achieve over 7% better accuracy on mechanical drawing tasks, thanks to the new MechVQA dataset and the specialized MechVL model.
Multimodal Large Language Models (MLLMs) have demonstrated significant achievements in general visual question answering (VQA) tasks. However, they remain brittle on mechanical engineering drawings, where high annotation density and weak domain knowledge, compounded by unreliable spatial relation reasoning under strict projection rules and geometric constraints, make decisive cues easy to miss and frequently lead to wrong answers. To bridge this gap, we introduce the first comprehensive mechanical drawing understanding dataset, MechVQA, created through a semi-automated construction and quality-control pipeline. MechVQA contains 3.3k high-density pictures with 21K question-answer pairs, spanning 10 different fine-grained tasks across three capability levels: Recognition, Reasoning, and Judging, providing a testbed to evaluate and improve MLLM understanding on real-world mechanical drawings. On top of MechVQA, we then develop the MechVL model through a multi-stage training paradigm, building a strong domain-specialized baseline. Extensive experimental results demonstrate that MechVL outperforms the strongest closed-source baseline by 7.57 percentage points on the MechVQA total score, significantly enhancing mechanical drawing understanding ability and providing a reusable foundation for deploying MLLMs in mechanical design and inspection scenarios.
