CerraData-4MM: A multimodal benchmark dataset on Cerrado for land use and land cover classification

2025-01-31

AI Summary

To address severe class imbalance and spectral confusion in land use/land cover (LULC) mapping of Brazil’s Cerrado biome, this work introduces the first open-source, multimodal remote sensing benchmark dataset tailored to the region, integrating co-registered Sentinel-1 SAR and Sentinel-2 MSI imagery (10 m resolution) for hierarchical two-level classification (7 and 14 classes). We propose a novel hierarchical labeling protocol and a weighted training framework that jointly optimizes minority-class representation and overall accuracy. Extensive experiments with U-Net and Vision Transformer (ViT) architectures demonstrate that ViT achieves macro-F1 of 57.60% and mIoU of 49.05% on level-1 classification, substantially improving minority-class discrimination. Our analysis further uncovers fundamental performance bottlenecks of multimodal modeling under extreme class imbalance. The dataset, models, and source code are fully open-sourced.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

The Cerrado faces increasing environmental pressures, necessitating accurate land use and land cover (LULC) mapping despite challenges such as class imbalance and visually similar categories. To address this, we present CerraData-4MM, a multimodal dataset combining Sentinel-1 Synthetic Aperture Radar (SAR) and Sentinel-2 MultiSpectral Imagery (MSI) with 10m spatial resolution. The dataset includes two hierarchical classification levels with 7 and 14 classes, respectively, focusing on the diverse Bico do Papagaio ecoregion. We highlight CerraData-4MM's capacity to benchmark advanced semantic segmentation techniques by evaluating a standard U-Net and a more sophisticated Vision Transformer (ViT) model. The ViT achieves superior performance in multimodal scenarios, with the highest macro F1-score of 57.60% and a mean Intersection over Union (mIoU) of 49.05% at the first hierarchical level. Both models struggle with minority classes, particularly at the second hierarchical level, where U-Net's performance drops to an F1-score of 18.16%. Class balancing improves representation for underrepresented classes but reduces overall accuracy, underscoring the trade-off in weighted training. CerraData-4MM offers a challenging benchmark for advancing deep learning models to handle class imbalance and multimodal data fusion. Code, trained models, and data are publicly available at https://github.com/ai4luc/CerraData-4MM.

Problem

• Cerrado region
• land use mapping
• category imbalance

Innovation

• Multimodal Dataset
• Land Use Classification
• Imbalanced Categories