Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper introduces Kernel Token Contradiction (KTC), a novel method for claim-level uncertainty quantification (UQ) in large language models (LLMs) that leverages a positive semi-definite kernel to integrate the LLM's conditional distribution with a token contradiction score. KTC achieves significant computational efficiency, boasting over an 8.2x speedup compared to state-of-the-art GPU methods and over a 65x speedup against comparable CPU-only methods, while maintaining high accuracy across multiple benchmarks and languages. The results indicate that KTC not only matches but often exceeds the performance of existing UQ methods, enabling practical real-time monitoring of LLM outputs in production environments.
KTC achieves over 8x faster claim uncertainty quantification without sacrificing accuracy, making real-time LLM monitoring feasible.
Claim-level Uncertainty Quantification (UQ) aims to mitigate the lack of reliability of Large Language Models (LLMs) by evaluating the factuality of each claim in their outputs. We introduce Kernel Token Contradiction (KTC), a lightweight approach to compute claim-level UQ under realistic white-box conditions. KTC represents the candidate tokens involved in LLM generation as a positive semi-definite kernel that integrates both the LLM's conditional distribution and a token contradiction score. We then use the Von Neumann entropy to quantify the uncertainty of this kernel. To estimate token contradiction, we develop a new approach based on frequency statistics from the Wikipedia corpus. Although CPU-only, our approach achieves over an 8.2x speedup compared to state-of-the-art GPU-accelerated methods based on cross-encoders, and over a 65x speedup compared to CPU-only methods with comparable performance. Our evaluation spans two benchmarks across four European languages and 16 different models. KTC not only matches the average performance of existing methods but also outperforms them in high-precision regimes. This combination of computational efficiency and accuracy makes real-time monitoring of LLM outputs practical in production.