Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces CHAIR, a supervised framework that detects hallucinations in instruction-tuned LLMs by analyzing internal logits from each token's layers. By extracting features such as maximum, minimum, mean, standard deviation, and slope from these logits, CHAIR enhances detection accuracy, particularly in zero-shot scenarios, as evidenced by experiments on the TruthfulQA and MMLU datasets. The findings suggest that leveraging internal representations can not only improve hallucination detection but also inform the development of advanced decoding strategies to enhance text quality and factuality.
CHAIR reveals that analyzing internal logits can significantly boost hallucination detection accuracy in LLMs, especially in zero-shot contexts.
In this work, we introduce CHAIR (Classifier of Hallucination As ImproveR), a supervised framework for detecting hallucinations by analyzing internal logits from each layer of every token. Our method extracts a compact set of features such as maximum, minimum, mean, standard deviation, and slope-from the token logits across all layers, enabling effective hallucination detection without overfitting. Experiments on TruthfulQA and MMLU datasets demonstrate that CHAIR significantly improves detection accuracy, particularly in zero-shot scenarios, showcasing its robustness and generalizability. Beyond hallucination detection, CHAIR highlights the potential of using internal representations for designing advanced decoding strategies. By leveraging patterns in logits, we suggest that more sophisticated models and adaptive decoding methods could further reduce hallucinations and enhance text completion quality. CHAIR not only offers a practical solution for detecting hallucinations but also lays the groundwork for exploring richer representations in LLMs to improve their factuality and coherence.