Text-Aided Multi-Modal Panoptic Symbol Spotting for CAD Floor Plan Drawings

2026-07-14

AI Summary

Existing methods for CAD symbol recognition often overlook the deep semantic content of textual annotations, thereby limiting holistic detection performance. To address this limitation, this work proposes TextCAD, a multimodal framework that jointly models graphical primitives and textual annotations through a Type-Attribute Correlation Encoder (TACE) and a multi-level semantic alignment mechanism. This approach effectively integrates their compositional semantics and hierarchical structure. By incorporating primitive downsampling and cross-modal semantic injection strategies, the proposed method achieves state-of-the-art symbol recognition accuracy on real-world architectural drawing datasets, significantly outperforming existing approaches.

Abstract

Computer-Aided Design (CAD) floor plan drawings contain both graphical primitives and textual annotations, which provide complementary geometric and semantic cues for intelligent design understanding. Among CAD analysis tasks, panoptic symbol spotting has become increasingly important with the growing demand for industrial digitalization and deep learning-based automation. However, most existing methods remain primarily primitive-centric and underexploit textual annotations, despite their critical semantic value. Even the few text-aware approaches often treat annotations only superficially, without properly modeling complex syntax and hierarchical semantics of CAD annotations, which leads to semantic loss and suboptimal spotting performance. To address these limitations, we propose TextCAD, a multimodal framework that jointly models graphical primitives and textual annotations for panoptic symbol spotting. Specifically, we design a Type-Attribute Correlation Encoder (TACE) to explicitly encode the compositional semantics within annotations by jointly modeling their types and attributes. We further introduce a Semantic Hierarchy Alignment framework with Multi-level Semantic Filtering (MSF) and primitive downsampling, which adaptively aligns annotation semantics with graphical primitives at different semantic levels and enables accurate cross-modal semantic injection and fusion. Experiments on real-world building-design datasets show that TextCAD effectively improves symbol spotting performance and achieves state-of-the-art results.

Problem

• panoptic symbol spotting
• CAD floor plan
• textual annotations
• semantic modeling
• multi-modal

Innovation

• multi-modal learning
• text-graphic fusion
• panoptic symbol spotting
• semantic hierarchy alignment
• CAD floor plan understanding