FloorplanVLM: A Vision-Language Model for Floorplan Vectorization

2026-02-06

AI Summary

This work proposes a novel end-to-end method for converting raster floorplans into engineering-grade vector graphics that satisfy complex topological and geometric constraints. By reframing vectorization as an image-conditioned sequence modeling task, the approach directly outputs a JSON sequence encoding the global structural layout, enabling precise holistic representation of non-Manhattan elements such as slanted walls and arcs. Adopting a “pixel-to-sequence” paradigm, the method circumvents the fragmentation and heuristic fragility of conventional pipelines. Trained on two newly curated datasets—Floorplan-2M and Floorplan-HQ-300K—using a vision-language model architecture with supervised fine-tuning and Group Relative Policy Optimization (GRPO), the model achieves a 92.52% exterior wall IoU on the FPBench-2K benchmark, demonstrating substantial improvements in structural validity and generalization capability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Deep Neural Architectures and Foundation Models

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Converting raster floorplans into engineering-grade vector graphics is challenging due to complex topology and strict geometric constraints. To address this, we present FloorplanVLM, a unified framework that reformulates floorplan vectorization as an image-conditioned sequence modeling task. Unlike pixel-based methods that rely on fragile heuristics or query-based transformers that generate fragmented rooms, our model directly outputs structured JSON sequences representing the global topology. This'pixels-to-sequence'paradigm enables the precise and holistic constraint satisfaction of complex geometries, such as slanted walls and curved arcs. To support this data-hungry approach, we introduce a scalable data engine: we construct a large-scale dataset (Floorplan-2M) and a high-fidelity subset (Floorplan-HQ-300K) to balance geometric diversity and pixel-level precision. We then employ a progressive training strategy, using Supervised Fine-Tuning (SFT) for structural grounding and quality annealing, followed by Group Relative Policy Optimization (GRPO) for strict geometric alignment. To standardize evaluation on complex layouts, we establish and open-source FPBench-2K. Evaluated on this rigorous benchmark, FloorplanVLM demonstrates exceptional structural validity, achieving $\textbf{92.52%}$ external-wall IoU and robust generalization across non-Manhattan architectures.

Problem

• floorplan vectorization
• raster-to-vector conversion
• geometric constraints
• topology representation
• engineering-grade vector graphics

Innovation

• floorplan vectorization
• vision-language model
• sequence modeling
• geometric constraint satisfaction
• progressive training