Search papers, labs, and topics across Lattice.
This paper introduces Verification Autonomy Levels (VAL), a meta-standard that categorizes verification schemes for large language models (LLMs) based on the source of their verification specifications and the guarantees they provide. By distinguishing between levels from L0 (LLM self-declaration) to L5 (impossible in unrestricted cases), the authors highlight a critical completeness blind spot in existing verification methods, where empirical verifiers can confirm correctness but cannot ensure that no errors were overlooked. The findings underscore the limitations of current verification approaches across various domains, including symbolic mathematics and medical diagnosis, and clarify misconceptions in the verification literature regarding granularity and risk.
Verification schemes for LLMs reveal a critical blind spot: while they can confirm correctness, they often miss potential errors entirely.
Large language models (LLMs) are increasingly paired with verifiers (step checkers, self-consistency filters, tool-based fact checkers, formal proof assistants) that claim to detect the model's errors. Yet the verification literature uses the word"level"to mean at least five different things: verification granularity, concept abstraction, risk tier, system-stack layer, and the epistemic source of the ground truth. We propose Verification Autonomy Levels (VAL), a meta-standard classifying verification schemes along a single axis: where does the verification spec come from, and what does the verdict guarantee? VAL ranges from L0 (LLM self-declaration, no deterministic anchor) through L2 (objective ground truth, correctness only) to L3/L4 (decidable systems with single-property or domain-level completeness), with L5 impossible in the unrestricted case. Central to VAL is the completeness blind spot: substitution- and sampling-based verifiers can confirm that proposed candidates hold, but cannot prove that no candidate was missed. We further identify a dichotomy the literature has not stated: completeness is reachable only for formally specifiable properties, while empirical open-world verification (fact-checking, diagnosis) caps at anchored correctness (L2). We document this across four domains (symbolic mathematics, behavior monitoring, medical diagnosis, and code generation) and in the strongest existing formal-verification baseline, whose authors note the verifier"focuses on the correctness of each step."We show the levels of granularity, concept hierarchy, risk, and system stack are orthogonal to VAL, resolving a systematic conflation across 17 surveyed papers. Code and full assessment are released as supplementary material.
