Search papers, labs, and topics across Lattice.
This paper evaluates 25 LLMs across four materials science tasks to understand their reliability and knowledge encoding. The study reveals that output modality significantly impacts model behavior: fine-tuning leads to consistent symbolic answers but inconsistent numerical predictions. The authors also find that extracting embeddings from intermediate transformer layers outperforms using text output for numerical regression, highlighting an "LLM head bottleneck," and observe significant performance variations (9-43%) in GPT models over 18 months, raising reproducibility concerns.
LLMs struggle to reliably predict numerical materials properties, even after fine-tuning, and their performance fluctuates wildly over time, casting doubt on their use in high-stakes scientific applications.
Large language models are increasingly applied to materials science, yet fundamental questions remain about their reliability and knowledge encoding. Evaluating 25 LLMs across four materials science tasks -- over 200 base and fine-tuned configurations -- we find that output modality fundamentally determines model behavior. For symbolic tasks, fine-tuning converges to consistent, verifiable answers with reduced response entropy, while for numerical tasks, fine-tuning improves prediction accuracy but models remain inconsistent across repeated inference runs, limiting their reliability as quantitative predictors. For numerical regression, we find that better performance can be obtained by extracting embeddings directly from intermediate transformer layers than from model text output, revealing an ``LLM head bottleneck,'' though this effect is property- and dataset-dependent. Finally, we present a longitudinal study of GPT model performance in materials science, tracking four models over 18 months and observing 9--43\% performance variation that poses reproducibility challenges for scientific applications.
MIT CSAIL