Search papers, labs, and topics across Lattice.
This paper addresses the issue of inaccuracies in statistical reporting generated by large language models (LLMs) by introducing a claim-locked reporting protocol that fixes the evidence source and numerical values prior to prose generation. By implementing this method, the authors demonstrate significant improvements in reproducibility, achieving a 37.4% and 20.5% increase in accuracy for fMRI functional-connectivity and randomized controlled trial reports, respectively. Human audits further validate the effectiveness of this approach in preserving the direction of claims and optimizing resource usage during report generation.
Claim-locked reporting boosts the accuracy of LLM-generated statistical reports by over 37%, ensuring that evidence integrity is maintained throughout the writing process.
Large language models (LLMs) can fluently verbalize statistical evidence, yet statistical reports can still drift numerical values, invert effect directions, or restate thresholded contrasts as categorical effects. We frame these failures as a control problem: the evidence-bearing content of a scientific report should be fixed by structured statistical results rather than sampled during prose generation. We therefore use cross-run reproducibility to stress-test whether report-visible numbers and claims are bound before prose generation. Existing controls operate at the text or slot level; a deterministic hybrid template reproduces only 61.1% of report-visible numerical content across seeds because the LLM still selects which findings and numbers the template renders. We propose claim-locked reporting, a provenance-before-prose protocol that fixes the evidence source, numbers, direction, and allowed language strength of each reportable claim before the LLM writes only connective prose. Across fMRI functional-connectivity reporting and randomized controlled trial reporting on Evidence Inference 2.0, claim-locked reporting improves reproducibility over the hybrid template by 37.4 and 20.5 points, respectively. Blinded human audits support the observed direction-preservation and governance trends. In an fMRI cost analysis with DeepSeek, claim-locked reporting also yields the lowest observed token use and median generation latency.
