Think Inside the Chunk: RegulaRAG for Regulation-Compliant Scenario Generation using LLMs: A Case Study of UN Regulation No. 152

2026-08-17

AI Summary

This study addresses the challenge of large language models in accurately comprehending hierarchical regulations within long texts by proposing RegulaRAG, a novel pipeline that enhances semantic units through SmartChunking and graph traversal. Integrating retrieval, reranking, and generation, the system synthesizes compliance test scenarios while employing a progressive search strategy for parameter optimization. Evaluated on the UN R152 dataset, RegulaRAG achieves a Meta-Score of 82.99, surpassing the suboptimal baseline by 43% with token consumption limited to 14k–25k. Furthermore, the method demonstrates robust performance under corpus expansion, enabling efficient and precise automatic generation of regulatory compliance testing scenarios.

Abstract

Generating regulation-compliant test scenarios is essential for validating safety-critical automotive systems, yet Large Language Models (LLMs) struggle to ground outputs in long, hierarchical standards. We present RegulaRAG, a Retrieval-Augmented Generation (RAG) pipeline that couples SmartChunking, reference-aware enrichment of paragraphs and tables via graph traversal, with Smart Retrieve & Rerank over these enriched units. To test our system, we evaluate on a manually curated dataset covering all scenarios in UN Regulation No. 152 (AEBS). Our study comprises: (i) a three-step progressive search that identifies near-optimal retrieval parameters without exhaustive grid search; (ii) head-to-head comparisons against five baseline RAG systems; and (iii) a robustness stress test that scales the source corpus with distractor content. Outputs are evaluated using a customized penalized scoring metric. Across all experiments, RegulaRAG achieves the highest average Meta-Score (82.99), outperforming the next-best system by 43% (NoRAG: 57.94), while operating at 14k-25k tokens per query versus up to 500k for graphcentric baselines. It maintains strong performance, remaining stable even as the number of regulatory sources grows, whereas competing RAG systems degrade sharply in both quality and robustness.

Problem

• Regulation-compliant scenario generation
• Large Language Models
• Hierarchical standards grounding
• Automotive safety validation

Innovation

• RegulaRAG
• SmartChunking
• Reference-aware Enrichment
• Smart Retrieve & Rerank
• Progressive Search