Search papers, labs, and topics across Lattice.
The paper introduces SpecVQA, a new benchmark for evaluating MLLMs on understanding scientific spectra, comprising 620 figures and 3100 QA pairs across 7 spectrum types extracted from scientific literature. To improve performance, they propose a spectral data sampling and interpolation reconstruction approach to reduce token length while preserving curve characteristics. Experiments show that this approach improves performance on the SpecVQA benchmark, and the benchmark itself reveals the limitations of current MLLMs in scientific spectral understanding.
Current multimodal LLMs struggle to understand scientific spectra, but a new benchmark and data processing technique could change that.
Spectra are a prevalent yet highly information-dense form of scientific imagery, presenting substantial challenges to multimodal large language models (MLLMs) due to their unstructured and domain-specific characteristics. Here we introduce SpecVQA, a professional scientific-image benchmark for evaluating multimodal models on scientific spectral understanding, covering 7 representative spectrum types with expert-annotated question-answer pairs. The aim comprises two aspects: spectra scientific QA evaluation and corresponding underlying task evaluation. SpecVQA contains 620 figures and 3100 QA pairs curated from peer-reviewed literature, targeting both direct information extraction and domain-specific reasoning. To effectively reduce token length while preserving essential curve characteristics, we propose a spectral data sampling and interpolation reconstruction approach. Ablation studies further confirm that the approach achieves substantial performance improvements on the proposed benchmark. We test the capability of prominent MLLMs in scientific spectral understanding on our benchmark and present a leaderboard. This work represents an essential step toward enhancing spectral understanding in multimodal large models and suggests promising directions for extending visual-language models to broader scientific research and data analysis.
