Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
LitXBench is introduced as a framework for benchmarking methods that extract full experimental measurements from scientific literature, addressing the need for improved property prediction models in materials science. The framework includes LitXAlloy, a dense benchmark of 1426 measurements from 19 alloy papers, stored as Python objects for enhanced auditability and data validation. Experiments show that advanced language models like Gemini 3.1 Pro Preview significantly outperform existing extraction pipelines by up to 0.37 F1, highlighting the importance of capturing processing steps in material definitions.
Gemini 3.1 Pro Preview can extract scientific experimental data with significantly higher fidelity than existing pipelines, but still struggles to link processing steps to material properties.
Aggregating experimental data from papers enables materials scientists to build better property prediction models and to facilitate scientific discovery. Recently, interest has grown in extracting not only single material properties but also entire experimental measurements. To support this shift, we introduce LitXBench, a framework for benchmarking methods that extract experiments from literature. We also present LitXAlloy, a dense benchmark comprising 1426 total measurements from 19 alloy papers. By storing the benchmark's entries as Python objects, rather than text-based formats such as CSV or JSON, we improve auditability and enable programmatic data validation. We find that frontier language models, such as Gemini 3.1 Pro Preview, outperform existing multi-turn extraction pipelines by up to 0.37 F1. Our results suggest that this performance gap arises because extraction pipelines associate measurements with compositions rather than the processing steps that define a material.