Search papers, labs, and topics across Lattice.
The paper introduces MaterEval, a Knowledge-Augmented Preference Signals framework, to train LLMs for reliable materials evaluation by contrasting "informed judgments" (following expert rules with evidence) against "blind guesses" (rule-removed). This preference data guides LLMs to emulate expert reasoning without external knowledge retrieval. Applied to high-entropy alloy assessment, MaterEval enables small, open-source LLMs to achieve accuracy, consistency, and evidence discrimination comparable to rule-based closed-source LLMs, demonstrating the transfer of expert knowledge into learnable preference signals.
Small, open-source LLMs can rival the materials evaluation capabilities of proprietary systems, simply by learning to distinguish between expert-informed judgments and uninformed guesses.
As candidate generation and high-throughput experimentation advance, the primary bottleneck in materials discovery is shifting from property prediction to making reliable evaluations among massive candidate sets. We propose a Knowledge-Augmented Preference Signals Framework, MaterEval, that automatically produces, for the same candidate, two evaluations: an informed judgment that follows expert rules and provides supporting evidence, and a rule-removed blind guess. By pairing the two evaluations as preference data, we guide general-purpose large language models (LLMs), originally lacking materials-specific criteria, from intuitive judgment toward reliable evaluation supported by explicit evidence. To balance throughput, cost, and reliability, we further introduce a fast-slow reasoning scheme that decouples large-scale rapid screening from in-depth review on a small subset. Using high-entropy alloy (HEA) assessment as a case study, we show that, without external retrieval and relying solely on internalized capabilities, small open-source LLMs achieve substantial gains in accuracy, conclusion consistency, and evidence discrimination, approaching the performance of rule-based closed-source LLMs. These results demonstrate that expert rules can be systematically transformed into learnable preference signals, enabling a low-cost and deployable evaluation module for autonomous materials discovery loops.
