Search papers, labs, and topics across Lattice.
This paper introduces a novel framework for enhancing the reasoning capabilities of Large Language Models (LLMs) by structuring their outputs as weighted Directed Acyclic Graphs (DAGs) that reflect the consensus of multiple reasoning traces. By aggregating these DAGs, the method not only improves accuracy on various high-stakes tasks but also provides an inspectable graph that reveals the reasoning process behind the model's conclusions. The approach outperforms traditional majority-vote baselines, achieving up to a 3.1% accuracy increase on narrative multi-hop reasoning tasks while correlating ensemble weights with expert evaluations of reasoning quality.
Consensus reasoning through structured DAG aggregation reveals hidden insights in LLM outputs, outperforming traditional methods and providing a transparent view of model reasoning.
Large Language Models (LLMs) explore problems through chain-of-thought, but this exploration is buried in unstructured prose. On high-stakes tasks, users cannot tell which steps are well-supported, which alternatives were seriously considered, or how the final conclusion compares to those the model discarded. We propose a framework that ensembles the reasoning structure, not just the answers, of multiple LLMs by weighted merging of Directed Acyclic Graphs (DAGs) extracted from reasoning chains. We weight each step by how many traces independently attest to it, to return"Consensus Reasoning". Across six benchmarks spanning statutory interpretation, graduate-level science, narrative multi-hop reasoning, and first-order logic, our ensemble outperforms a matched-budget majority-vote baseline, with a maximum accuracy gain of 3.1% on MuSR-MM (narrative multi-hop reasoning). On a single model, the framework matches or exceeds self-consistency at the same trace budget while additionally exposing an inspectable consensus reasoning graph. Ensemble weights correlate with LLM-judge rankings of reasoning quality at Spearman $\rho = 0.30$-$0.51$, and consensus subgraphs are preferred over alternatives leading to the majority-vote answer in 54.4-65.4% of head-to-head comparisons across five of six datasets. We observe that our framework can also be used to analyze diverse reasoning perspectives for a problem.
