Symmetrization of Loss Functions for Robust Training of Neural Networks in the Presence of Noisy Labels

2026-05-19

AI Summary

Label noise significantly degrades model performance, necessitating the design of robust loss functions. This work proposes a symmetric decomposition–based framework for constructing multiclass losses and establishes, for the first time, that the unhinged loss is the unique multiclass loss satisfying both convexity and symmetry. It further reveals the central role of the unhinged loss as the local linear approximation of any symmetric loss. Building on this insight, the authors introduce two families of interpolation losses—SGCE and alpha-MAE—with tunable smoothness controlled via β-smoothness, thereby balancing noise robustness and optimization stability. Empirical evaluations on standard noisy-label benchmarks demonstrate that the proposed losses match or surpass the performance of current state-of-the-art robust loss functions.

Abstract

Labeling a training set is often expensive and susceptible to errors, making the design of robust loss functions for label noise an important problem. The symmetry condition provides theoretical guarantees for robustness to such noise. In this work, we study a symmetrization method arising from the unique decomposition of any multi-class loss function into a symmetric component and a class-insensitive term. In particular, symmetrizing the cross-entropy loss leads to a linear multi-class extension of the unhinged loss. Unlike in the binary case, the multi-class version must have specific coefficients in order to satisfy the symmetry condition. Under suitable assumptions, we show that this multi-class unhinged loss is the unique convex multi-class symmetric loss. We also show that it has a fundamental local role: the linear approximation of any symmetric loss around score vectors with equal components is equivalent to the multi-class unhinged loss. We then introduce SGCE and alpha-MAE, two loss functions that interpolate between the multi-class unhinged loss and the Mean Absolute Error while allowing control of the beta-smoothness of the loss. Experiments on standard noisy-label benchmarks show competitive performance compared with existing robust loss functions.

Problem

• noisy labels
• robust loss functions
• symmetry condition
• neural network training
• label noise

Innovation

• symmetrization
• noisy labels
• multi-class unhinged loss
• robust loss functions
• beta-smoothness