Search papers, labs, and topics across Lattice.
This paper introduces a unified benchmark for span-level hallucination detection that encompasses various structured inputs, including code and tool outputs, alongside traditional natural-language documents. The authors validate their approach by injecting localized hallucinations into grounded answers and assessing performance using a fine-tuned Qwen3.5-2B model, which achieves a span-F1 score of 0.689 on the unified test set. Notably, this model significantly outperforms existing methods, including LettuceDetect-large, highlighting its effectiveness in detecting hallucinations across diverse input types.
Span-level hallucination detection can now effectively address structured inputs like code and tool outputs, not just natural language, revealing a critical gap in current RAG evaluations.
Hallucination detection for retrieval-augmented generation (RAG) is usually evaluated on natural-language document evidence. However, grounded generation systems increasingly rely on structured inputs: source code, developer-tool output, markdown documents, tables, and repository metadata. We introduce a unified benchmark for span-level hallucination detection over code, tool output, structured documents, and existing natural-language RAG datasets. The benchmark is built by starting from grounded correct answers, injecting localized hallucinations with exact character labels, and validating the code test split with evidence-based review. Our fine-tuned Qwen3.5-2B detector reaches 0.689 span-F1 on the unified test set and 0.60 on the code-agent source, where it substantially outperforms LettuceDetect-large (0.17) and the strongest zero-shot LLM judges we evaluated (at most 0.22). The same model remains competitive on established natural-language benchmarks, with 81.8 RAGTruth example-F1 and 0.724 English PsiloQA IoU.
