BIM-Edit: Benchmarking Large Language Models for IFC-Based Building Information Modeling

2026-06-18

AI Summary

This study addresses the lack of evaluation benchmarks for assessing large language models’ ability to edit existing building information models (BIM) while preserving semantic and topological integrity. To bridge this gap, the authors propose BIM-Edit, the first natural language editing benchmark tailored to IFC-based BIMs, encompassing direct, spatial, and topological editing instructions. They further introduce a comprehensive three-dimensional evaluation framework that jointly assesses geometric, semantic, and topological fidelity. An empirical evaluation of leading large language models across 324 tasks reveals significant limitations: the best-performing model achieves only a 49.5% average score, with a full-correctness rate below 3.4%. These findings underscore a substantial performance gap between current capabilities and practical engineering requirements, thereby establishing a foundational benchmark for structured BIM editing.

Abstract

Large language models (LLMs) are increasingly applied to computer-aided design (CAD) to generate design artifacts from textual instructions. In engineering practice, this requires more than creating new geometry, models must also understand existing scenes, edit them correctly, and preserve semantics and relations. However, many CAD benchmarks focus on creating new models rather than editing existing ones, and mostly evaluate geometric correctness. We introduce BIM-Edit, a benchmark for evaluating LLMs on natural-language editing of Building Information Models (BIM) represented in the Industry Foundation Classes (IFC) format. BIM provides a challenging testbed because building models encode geometry together with semantic and relational structure. BIM-Edit contains 324 editing tasks spanning 11 realistic building models and 36 synthetic scenes. Tasks are expressed using three instruction categories - direct, spatial, and topological - covering both explicit and scene-grounded edits. We evaluate outputs along three dimensions: geometric accuracy, semantic validity, and topological consistency. Across evaluated LLMs, the best-performing model achieves only 49.5% average score across the three metrics, and no model fully solves more than 3.4% of tasks. These results demonstrate a substantial gap between current LLM capabilities and the requirements of structured engineering design workflows.

Problem

• Building Information Modeling
• Large Language Models
• IFC
• Model Editing
• Semantic Consistency

Innovation

• BIM-Edit
• Large Language Models
• IFC
• semantic editing
• topological consistency