A class-driven hierarchical ResNet for classification of multispectral remote sensing images

2025-10-09

AI Summary

To address weak fine-grained discrimination and difficulty in expanding to novel classes under limited samples in multitemporal multispectral remote sensing classification, this paper proposes a class-driven hierarchical ResNet model. The model employs a multi-branch residual architecture integrated with hierarchical penalty graph constraints to jointly optimize coarse-to-fine classification paths. A hierarchical loss function and a coarse-to-fine training strategy are designed to enhance semantic consistency and improve minority-class accuracy. Leveraging transfer learning and fine-tuning, the model is trained end-to-end on 12-time-series Sentinel-2 images. Experiments demonstrate strong generalization across both macro- and micro-level categories, with significant gains in fine-grained classification accuracy—particularly for scarce classes. The framework supports modular extension and task-incremental learning in low-resource settings.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Remote Sensing / Geospatial AI
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for heterogeneous, signed, attributed, multi-relational, temporal, higher-order, and annotated Web-related graphs
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

This work presents a multitemporal class-driven hierarchical Residual Neural Network (ResNet) designed for modelling the classification of Time Series (TS) of multispectral images at different semantical class levels. The architecture consists of a modification of the ResNet where we introduce additional branches to perform the classification at the different hierarchy levels and leverage on hierarchy-penalty maps to discourage incoherent hierarchical transitions within the classification. In this way, we improve the discrimination capabilities of classes at different levels of semantic details and train a modular architecture that can be used as a backbone network for introducing new specific classes and additional tasks considering limited training samples available. We exploit the class-hierarchy labels to train efficiently the different layers of the architecture, allowing the first layers to train faster on the first levels of the hierarchy modeling general classes (i.e., the macro-classes) and the intermediate classes, while using the last ones to discriminate more specific classes (i.e., the micro-classes). In this way, the targets are constrained in following the hierarchy defined, improving the classification of classes at the most detailed level. The proposed modular network has intrinsic adaptation capability that can be obtained through fine tuning. The experimental results, obtained on two tiles of the Amazonian Forest on 12 monthly composites of Sentinel 2 images acquired during 2019, demonstrate the effectiveness of the hierarchical approach in both generalizing over different hierarchical levels and learning discriminant features for an accurate classification at the micro-class level on a new target area, with a better representation of the minoritarian classes.

Problem

• Classifying multispectral remote sensing time series images hierarchically
• Improving discrimination of classes at different semantic detail levels
• Training modular architecture with limited samples for new classes

Innovation

• Hierarchical ResNet with additional branches for multi-level classification
• Hierarchy-penalty maps to enforce coherent class transitions
• Modular architecture enabling fine-tuning for new classes