Mask-to-Height: A YOLOv11-Based Architecture for Joint Building Instance Segmentation and Height Classification from Satellite Imagery

2025-10-31

AI Summary

To address the joint modeling challenge of building instance segmentation and discrete height classification in satellite imagery, this paper proposes a multi-task collaborative framework built upon YOLOv11. We extend the YOLOv11 architecture by integrating a multi-scale feature enhancement module and parallel heads for instance segmentation and discrete height classification, explicitly capturing building geometric structure and hierarchical semantics. The design effectively mitigates occlusion, irregular shape ambiguity, and class imbalance—particularly improving robustness for tall buildings. Evaluated on the DFC2023 Track 2 dataset, our method achieves 60.4% mAP@50 and 38.3% mAP@50–95 for instance segmentation, while maintaining stable height classification accuracy and real-time inference speed. It outperforms existing state-of-the-art methods in overall performance, delivering an efficient end-to-end solution for large-scale urban 3D reconstruction.

Technology Category

• Machine Learning: Multi-instance/Multi-view Learning
• Computer Vision: Segmentation
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Accurate building instance segmentation and height classification are critical for urban planning, 3D city modeling, and infrastructure monitoring. This paper presents a detailed analysis of YOLOv11, the recent advancement in the YOLO series of deep learning models, focusing on its application to joint building extraction and discrete height classification from satellite imagery. YOLOv11 builds on the strengths of earlier YOLO models by introducing a more efficient architecture that better combines features at different scales, improves object localization accuracy, and enhances performance in complex urban scenes. Using the DFC2023 Track 2 dataset -- which includes over 125,000 annotated buildings across 12 cities -- we evaluate YOLOv11's performance using metrics such as precision, recall, F1 score, and mean average precision (mAP). Our findings demonstrate that YOLOv11 achieves strong instance segmentation performance with 60.4% mAP@50 and 38.3% mAP@50--95 while maintaining robust classification accuracy across five predefined height tiers. The model excels in handling occlusions, complex building shapes, and class imbalance, particularly for rare high-rise structures. Comparative analysis confirms that YOLOv11 outperforms earlier multitask frameworks in both detection accuracy and inference speed, making it well-suited for real-time, large-scale urban mapping. This research highlights YOLOv11's potential to advance semantic urban reconstruction through streamlined categorical height modeling, offering actionable insights for future developments in remote sensing and geospatial intelligence.

Problem

• Joint building instance segmentation and height classification from satellite imagery
• Handling occlusions and complex building shapes in urban scenes
• Outperforming earlier multitask frameworks in accuracy and speed

Innovation

• YOLOv11 architecture for joint building segmentation and height classification
• Efficient multi-scale feature combination improves urban scene analysis
• Real-time large-scale mapping with enhanced occlusion handling capability