Search papers, labs, and topics across Lattice.
This paper introduces HalluTracer, a novel framework for detecting hallucinations in large language models by aggregating truthfulness signals across all layers of the model's forward pass. The authors demonstrate that existing white-box detectors fail to leverage the distributed truthfulness information, leading to suboptimal performance in identifying factual inaccuracies. HalluTracer significantly outperforms traditional methods across multiple models and benchmarks, achieving improvements of one to fourteen points by utilizing depth-averaging techniques to enhance signal clarity and reduce noise.
HalluTracer transforms hallucination detection by revealing that aggregating truth signals across model layers can dramatically enhance accuracy.
Even well-aligned large language models confidently generate factually incorrect text, making hallucination a persistent reliability risk in high-stakes deployments. These models nonetheless carry linearly separable truthfulness signals in their internal representations. Existing white-box detectors, however, collapse this evidence to isolated components or a single depth, discarding discriminative information distributed across the full forward pass. We introduce HalluTracer, a detection framework that reads and aggregates truthfulness evidence across every layer of the forward pass before the model emits any answer token. A geometric analysis reveals that the per-layer signals are weakly correlated, so that simple depth averaging suppresses layer-specific noise and captures nearly all linearly accessible information. Across six open-source language models and five hallucination benchmarks, HalluTracer consistently outperforms matched white-box baselines, with gains ranging from one to fourteen points. Collectively, our work recasts hallucination detection from a layer-selection problem into a depth-aggregation problem governed by the geometric sparsity of the truthfulness signal.
Microsoft Research