Neural Architecture Search of Sample Reweighting Networks for Complex Distribution Shift

2026-06-22

AI Summary

This work addresses the performance degradation of existing sample reweighting methods under the compound distribution shift arising from concurrent label noise and class imbalance, where inaccurate weight estimation limits effectiveness. To overcome this, the study introduces neural architecture search (NAS) into Meta-Weight-Net for the first time, automatically optimizing its network structure by jointly searching the number of hidden layers, nodes per layer, and the optimal intermediate-layer feature inputs. This approach transcends the conventional paradigm of loss-driven weight learning by incorporating architectural adaptability. Leveraging a tree-structured Parzen estimator-based NAS strategy, the proposed method significantly enhances the robustness of the reweighting mechanism. Experiments on CIFAR-10 and CIFAR-100 demonstrate substantial performance gains, validating the efficacy and superiority of integrating architecture search into sample reweighting under challenging distribution shifts.

Abstract

Sample reweighting is a major approach to addressing distribution shifts, such as label noise and class imbalance. Meta-Weight-Net (MW-Net) is a promising sample reweighting network that computes weights based on classification loss. Although MW-Net improves prediction performance under a single type of distribution shift using a simple neural network, its performance degrades when facing both label noise and class imbalance, where it is hard to determine appropriate weights solely from classification loss and using a simple network. In this study, we introduce neural architecture search to MW-Net to mitigate such performance degradation. Using the tree-structured Parzen estimator, we explore the optimal number of hidden layers and nodes and select the most suitable intermediate layer in the classification model to serve as the input for MW-Net. Experimental results on the CIFAR-10 and CIFAR-100 datasets that were modified to include both label noise and class imbalance demonstrate the effectiveness of neural architecture search for MW-Net.

Problem

• distribution shift
• sample reweighting
• label noise
• class imbalance
• neural architecture search

Innovation

• Neural Architecture Search
• Sample Reweighting
• Distribution Shift
• Meta-Weight-Net
• Tree-structured Parzen Estimator