T2D-Bench: Evidence-Gated Evaluation of LLM Outputs for Type 2 Diabetes Using a Multi-Layer Clinical-Lifestyle Knowledge Graph

2026-06-23

AI Summary

This study addresses the frequent lack of guideline adherence and evidence-based lifestyle support in large language model (LLM) outputs when generating recommendations for type 2 diabetes management. The authors present the first multi-layer clinical–lifestyle knowledge graph integrating American Diabetes Association (ADA) care standards, biomedical knowledge, and glycemic-related lifestyle mechanisms. They further propose an evidence-gating mechanism that verifies and revises LLM-generated content through traceable, evidence-backed pathways. Evaluation across 100 structured clinical scenarios reveals that 35% of GPT-4o-mini and 33% of GPT-4o responses fail evidence validation. The proposed approach explicitly identifies and corrects unsupported statements, significantly enhancing the clinical compliance of generated recommendations.

Abstract

Large language models (LLMs) can produce clinically fluent recommendations for type 2 diabetes while failing to satisfy guideline constraints or explicitly justify lifestyle-related glycemic claims. We present T2D-Bench, a reproducible benchmark and evidence-gated evaluation framework for testing whether LLM outputs satisfy explicit, graph-checkable evidence requirements. T2D-Bench is built on a multi-layer clinical-lifestyle knowledge graph that combines a biomedical spine (UMLS, DrugBank, SIDER), computable ADA Standards of Care rules, and lifestyle knowledge connected through a mechanistic bridge to glycemic laboratory effects. Across 100 structured vignettes spanning diagnosis, medication safety, and adversarial lifestyle conflicts, baseline outputs failed benchmark-defined evidence-path checks in 35% of cases for GPT-4o-mini and 33% for GPT-4o. The evidence gate detects unsupported omissions and uses constrained revision to bring outputs into verifier-level compliance with benchmark-defined evidence requirements. These results show that computable evidence constraints can make unsupported clinical omissions explicit, measurable, and correctable in diabetes-focused LLM outputs.

Problem

• large language models
• type 2 diabetes
• clinical guidelines
• evidence justification
• lifestyle recommendations

Innovation

• evidence-gated evaluation
• multi-layer knowledge graph
• computable clinical guidelines
• lifestyle-glycemic mechanistic bridge
• LLM verification