Search papers, labs, and topics across Lattice.
This study introduces a graph-based framework for analyzing reasoning paths in scientific fact-checking, comparing human experts and large language models (LLMs). By modeling explanations as reasoning graphs, the authors align human and LLM reasoning at the level of fallacy-specific sub-graphs, revealing how different models approach the same misinformation. The evaluation of 84 false claims demonstrates that while Qwen3-32B exhibits the lowest verdict failure rate, GPT-5 shows the highest alignment with human reasoning, highlighting the nuanced performance of each model in this context.
Qwen3-32B may be the most reliable at verdicts, but GPT-5 outshines it in mimicking human reasoning paths during scientific fact-checking.
Misinformation that cites legitimate papers can be especially harmful when it distorts what those studies actually report. While existing automatic fact-checking systems based on large language models (LLMs) can assess whether a model assigns an Incorrect verdict and can gen- erate explanations for that decision, they typi- cally do not indicate whether the model follows the same reasoning path as human experts or arrives at the verdict through a different but still valid path. In this work, we introduce a graph- based framework (typed reasoning graph) for comparing human and LLM reasoning paths in scientific fact-checking. Building on prior work on fallacious reasoning in biomedical misinformation, MISSCIPLUS (Glockner et al., 2025), we model each explanation as a rea- soning graph that links the false claim to the relevant study context, study findings, fallacy- supporting premises, and fallacy labels. This representation enables one-to-one alignment of human and LLM reasoning at the level of fallacy-specific sub-graphs. For non-human- aligned LLM paths, we validate grounding in the cited study, relevance to the claim, and suf- ficiency for the verdict. Using 84 false claims from MISSCIPLUS, we evaluate GPT-5, Claude Opus 4.7, and Qwen3-32B across prompt and evidence settings. Results show distinct perfor- mance dimensions: Qwen3-32B has the lowest verdict failure rate, GPT-5 the highest human alignment, and Claude Opus 4.7 weak verdict prediction but often valid reasoning in success- ful cases
