DenoMAE2.0: Improving Denoising Masked Autoencoders by Classifying Local Patches

2025-02-25

AI Summary

To address the insufficient representation robustness and generalization in semi-supervised wireless communication learning caused by high noise levels and limited labeled data, this paper proposes a dual-objective self-supervised framework integrating local patch classification with global reconstruction. We innovatively design a position-aware unmasked patch classification task, coupled with an enhanced positional encoding-based classification head within a Masked Autoencoder (MAE) architecture. Joint optimization of L1 reconstruction loss and cross-entropy classification loss simultaneously strengthens fine-grained local feature discriminability and global structural consistency. On the RadioML benchmark, our method achieves 11.83% and 16.55% higher constellation classification accuracy than DenoMAE; on a custom low-SNR dataset, it improves overall accuracy by 1.1%, significantly enhancing denoising quality and downstream task performance. This work establishes a methodological breakthrough in semi-supervised representation learning for wireless signals.

Technology Category

• Machine Learning: Semi-Supervised Learning
• Search and Optimization: Learning to Search
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

We introduce DenoMAE2.0, an enhanced denoising masked autoencoder that integrates a local patch classification objective alongside traditional reconstruction loss to improve representation learning and robustness. Unlike conventional Masked Autoencoders (MAE), which focus solely on reconstructing missing inputs, DenoMAE2.0 introduces position-aware classification of unmasked patches, enabling the model to capture fine-grained local features while maintaining global coherence. This dual-objective approach is particularly beneficial in semi-supervised learning for wireless communication, where high noise levels and data scarcity pose significant challenges. We conduct extensive experiments on modulation signal classification across a wide range of signal-to-noise ratios (SNRs), from extremely low to moderately high conditions and in a low data regime. Our results demonstrate that DenoMAE2.0 surpasses its predecessor, Deno-MAE, and other baselines in both denoising quality and downstream classification accuracy. DenoMAE2.0 achieves a 1.1% improvement over DenoMAE on our dataset and 11.83%, 16.55% significant improved accuracy gains on the RadioML benchmark, over DenoMAE, for constellation diagram classification of modulation signals.

Problem

• Improves denoising masked autoencoders
• Enhances local patch classification
• Optimizes semi-supervised wireless communication learning

Innovation

• Enhanced denoising masked autoencoder
• Local patch classification objective
• Position-aware classification for fine-grained features