Scene Graph-Guided Generative AI Framework for Synthesizing and Evaluating Industrial Hazard Scenarios

2025-11-17

AI Summary

In industrial safety, the scarcity of real-world hazardous scene images hinders visual model training. Method: This paper proposes a scene-graph-guided diffusion generation framework: (1) GPT-4o parses OSHA accident reports to construct structured scene graphs encoding objects, attributes, and spatial relations; (2) these graphs condition text-to-image diffusion models to synthesize photorealistic hazardous scenes adhering to authentic risk logic; (3) a novel Graph Score metric—based on visual question answering (VQA)—is introduced to quantify semantic fidelity and spatial plausibility of generated images. Contribution/Results: To our knowledge, this is the first end-to-end framework synthesizing structured hazardous scene images directly from accident text. Evaluated on four state-of-the-art generative models, it achieves significant improvements in realism and semantic consistency over CLIP- and BLIP-based baselines, as measured by the VQA Graph Score.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Natural Language Processing: Safety and Robustness
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Training vision models to detect workplace hazards accurately requires realistic images of unsafe conditions that could lead to accidents. However, acquiring such datasets is difficult because capturing accident-triggering scenarios as they occur is nearly impossible. To overcome this limitation, this study presents a novel scene graph-guided generative AI framework that synthesizes photorealistic images of hazardous scenarios grounded in historical Occupational Safety and Health Administration (OSHA) accident reports. OSHA narratives are analyzed using GPT-4o to extract structured hazard reasoning, which is converted into object-level scene graphs capturing spatial and contextual relationships essential for understanding risk. These graphs guide a text-to-image diffusion model to generate compositionally accurate hazard scenes. To evaluate the realism and semantic fidelity of the generated data, a visual question answering (VQA) framework is introduced. Across four state-of-the-art generative models, the proposed VQA Graph Score outperforms CLIP and BLIP metrics based on entropy-based validation, confirming its higher discriminative sensitivity.

Problem

• Generating realistic industrial hazard images for training safety detection models
• Overcoming data scarcity by synthesizing accident scenarios from OSHA reports
• Evaluating generated image realism and semantic accuracy using VQA metrics

Innovation

• Scene graph-guided generative AI synthesizes hazard images
• GPT-4o extracts hazard reasoning from OSHA reports
• Visual question answering framework evaluates generated realism