Search papers, labs, and topics across Lattice.
This study introduces DiagFlowBench, a novel dataset designed to evaluate how language models manage off-procedure inputs during grounded diagnostic dialogues. By analyzing 1,676 multi-turn conversations derived from 50 industrial diagnostic flowcharts, the research highlights significant variability in the performance of ten evaluated models, particularly in their tendency to select contextually inappropriate yet plausible responses. The findings underscore a critical vulnerability in grounding systems, where models may provide misleading advice that appears authoritative but is contextually incorrect.
Language models often choose contextually inadequate yet plausible responses, revealing a dangerous vulnerability in grounded diagnostic systems.
Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in procedural documentation to constrain them to approved steps. In practice, however, operator queries frequently stray from this path, requiring models to recognise out-of-scope inputs mid-conversation, a dynamic that current benchmarks rarely prioritise. We introduce DiagFlowBench, a dataset of 50 industrial diagnostic flowcharts from a consumer manufacturer converted into 1,676 multi-turn conversations that contrast compliant with out-of-scope utterances. Evaluating a panel of ten commercial and open-weight models reveals high variability in abstention rates, with models commonly selecting a real but contextually inadequate step rather than fabricating facts. The inherent plausibility and authority of this mapped but wrong advice exposes a challenging vulnerability for grounding systems.
