Search papers, labs, and topics across Lattice.
This study investigates the limitations of large language models (LLMs) in accurately reasoning about chemical entities, revealing that while deterministic, database-grounded verification can effectively correct errors, the primary challenge lies in the detection of these errors rather than their repair. By employing a tiered verifier that extracts claims and checks them against authoritative databases, the authors demonstrate a significant reduction in committed-formula errors from 22% to 4% with substantially fewer retrievals compared to traditional methods. The findings highlight that while grounding enhances accuracy, its effectiveness is contingent on the verifier's ability to detect errors in long-tail scenarios, underscoring the need for improved detection mechanisms.
Detection, not repair, is the critical bottleneck in verifying chemical reasoning with large language models, revealing a 22% to 4% error reduction through targeted verification.
Large language models confabulate chemical objects (molecular formulas, space groups, formation energies) in fluent reasoning traces, concentrated on long-tail entities where confidence is least trustworthy. Deterministic, database-grounded verification can catch and repair such errors without the coverage cost of blanket retrieval; the binding constraint, we find, is detection, not repair. Our tiered verifier extracts each checkable claim, checks it against authoritative databases and physics, and feeds the reference into a gated correction loop. Across four models and 528 condition-pinned prompts, gated correction cuts committed-formula error from 22% to 4% at $3.2\times$ fewer retrievals than blanket augmentation, beating a conversational oracle. Repair succeeds wherever a flag fires (80--97%); the bottleneck is in-loop detection recall. Grounding improves the final answer only when the verifier's scope reaches the deliverable (83% to 90%), and the lift appears only where extractable long-tail error exists: absent on near-ceiling physical constants, large on isotope half-lives (11% to 0%).
