Search papers, labs, and topics across Lattice.
This paper introduces PhantomBench, a comprehensive benchmark designed to evaluate language models' (LMs) propensity for generating hallucinated responses based on non-existent terms and entities. By assessing 21 models across various types and sizes, the authors reveal alarmingly high hallucination rates, with some models exhibiting up to 86.7% inaccuracies when confronted with non-existent concepts. The findings underscore the critical need for improved model awareness of their knowledge limitations, especially in high-stakes applications where reliance on LMs can lead to significant harm.
Staggering hallucination rates of up to 86.7% reveal that even advanced language models struggle to recognize non-existent concepts, highlighting a critical vulnerability in AI reliability.
Hallucinations, where language models (LMs) generate factually ungrounded responses, pose serious risks, as users tend to blindly rely on them. This is particularly concerning in high-stakes domains, where consequences of such model behavior can lead to significant harms. Despite notable progress in understanding hallucinations, it remains unclear how reliably these models can recognize the limits of their knowledge. We introduce PhantomBench, the first large-scale benchmark of its kind, comprising more than 60K non-existent terms and entities derived from real concepts across diverse domains. Using our benchmark, we evaluate a total of 21 models of various types and sizes. We show staggering hallucination rates across the board (with average rates as high as 86.7% in some cases), and note that even frontier models surprisingly fail to abstain on non-existent concepts, especially when the input presumes their existence. We then show that PhantomBench can serve as a proxy for studying model behavior on rare concepts for which models are more prone to hallucinate. We also provide a pipeline to construct PhantomBench, enabling scalable generation of non-existent concepts tailored to the specific needs of researchers and practitioners.
