Search papers, labs, and topics across Lattice.
The paper introduces OPENXRD, an open-book question answering pipeline for crystallography that leverages GPT-4.5 to generate concise, domain-specific reference texts. It addresses the challenge of copyright issues associated with scanned textbooks by creating AI-generated summaries to augment smaller models' knowledge of X-ray diffraction (XRD). Experiments on a dataset of 217 expert-level XRD questions demonstrate that models utilizing OPENXRD achieve significant accuracy improvements, especially those with limited prior crystallography training.
Smaller vision-language models can punch above their weight in scientific domains like crystallography, given the right AI-generated open-book support.
This work presents OPENXRD, an open-book pipeline designed for crystallography question answering, which integrates textual prompts with concise supporting content generated by GPT-4.5. Instead of using scanned textbooks, which may lead to copyright issues, OPENXRD generates compact, domain-specific references that help smaller models understand key concepts in X-ray diffraction (XRD). We evaluate OPENXRD on a well-defined set of 217 expert-level XRD questions by comparing different vision-language models, including GPT-4 and LLaVA-based frameworks such as Mistral, LLaMA, and QWEN, under both closed-book (without supporting material) and open-book (with supporting material) conditions. Our experimental results show significant accuracy improvements in models that use the GPT-4.5-generated summaries, particularly those with limited prior training in crystallography. OPENXRD uses knowledge from larger models to fill knowledge gaps in crystallography and shows that AI-generated texts can help smaller models reason more effectively in scientific tasks. While the current version of OPENXRD focuses on text-based inputs, we also explore future extensions such as adding real crystal diagrams or diffraction patterns to improve interpretation in specialized materials science contexts. Overall, OPENXRD shows that specialized open-book systems can be useful in materials science and provides a foundation for broader natural language processing (NLP) tools in critical scientific fields.
