Search papers, labs, and topics across Lattice.
This paper introduces Trajel, a dataset and evaluation framework for identifying trajectory-level hallucinations in multi-agent industrial workflows, using a novel five-type hallucination taxonomy grounded in expert-annotated traces from AssetOpsBench. Benchmarking supervised detection models reveals that existing benchmarks miss many common failure modes, with nearly half of hallucinated trajectories exhibiting multiple hallucination types simultaneously. Trajectory-aware detection outperforms post-hoc verification, highlighting the necessity of taxonomy-grounded evaluation for safer agent deployment.
Hallucinations in multi-agent LLM workflows are far more complex and frequent than previously measured, with nearly half involving multiple types simultaneously, demanding a shift from final-answer checks to trajectory-aware auditing.
Large Language Models (LLMs) are increasingly deployed as autonomous agents that reason, use tools, and act over multiple steps. Yet most hallucination benchmarks still evaluate only the final output, missing failures that originate in intermediate Thought-Action-Observation steps. We present Trajel, a dataset and evaluation framework for auditing trajectory-level hallucinations in multi-agent industrial workflows. Trajel introduces a five-type hallucination taxonomy (factual, referential, logical, procedural, and scope-based) over expert-annotated agent traces from AssetOpsBench. We benchmark supervised detection models at the subtask, trajectory, and long-context levels. Our results show that the most common failure modes are missed by existing benchmarks, that nearly half of hallucinated trajectories involve multiple types at once, and that automated detectors with high binary accuracy still misclassify the subtlest types. Trajectory-aware detection significantly outperforms standard post-hoc verification, making taxonomy-grounded evaluation necessary for safer agentic deployment.
