DiagFlowBench: Evaluating How Language Models Handle Off-Procedure Inputs in Grounded Diagnostic Dialogue

2026-06-16

AI Summary

This study addresses the challenge that current language models struggle to reliably recognize and refuse out-of-protocol inputs in dialogue, often generating plausible yet incorrect recommendations. Focusing on industrial diagnostic scenarios, this work presents the first systematic investigation into models’ ability to abstain from responding to protocol-deviating queries. The authors construct a benchmark dataset comprising 1,676 multi-turn dialogues derived from 50 industrial diagnostic flowcharts, incorporating both compliant and out-of-protocol inputs. Through structured protocol transformation, dialogue synthesis, and context relevance evaluation, experiments on ten prominent commercial and open-source models reveal consistently low refusal rates, with models frequently producing factually accurate but procedurally inappropriate steps. These findings expose critical limitations in existing grounding mechanisms when operating under strict procedural constraints.

Abstract

Language models increasingly serve as advisory systems in maintenance operations. To prevent hallucination, recent systems ground these models in procedural documentation to constrain them to approved steps. In practice, however, operator queries frequently stray from this path, requiring models to recognise out-of-scope inputs mid-conversation, a dynamic that current benchmarks rarely prioritise. We introduce DiagFlowBench, a dataset of 50 industrial diagnostic flowcharts from a consumer manufacturer converted into 1,676 multi-turn conversations that contrast compliant with out-of-scope utterances. Evaluating a panel of ten commercial and open-weight models reveals high variability in abstention rates, with models commonly selecting a real but contextually inadequate step rather than fabricating facts. The inherent plausibility and authority of this mapped but wrong advice exposes a challenging vulnerability for grounding systems.

Problem

• grounded dialogue
• out-of-scope inputs
• diagnostic conversation
• language model hallucination
• procedural grounding

Innovation

• grounded dialogue
• out-of-scope detection
• diagnostic flowcharts
• language model hallucination
• procedural grounding