Label Distribution Learning with Biased Annotations by Learning Multi-Label Representation

2025-02-03

AI Summary

In label distribution learning (LDL), human annotation biases distort soft labels, degrading model performance. Method: This paper proposes a novel “degradation–reconstruction” paradigm: first degrading noisy soft labels into robust multi-hot hard labels, then reconstructing the underlying true label distribution via low-rank multi-label representation learning. Unlike conventional approaches imposing low-rank assumptions on the observation matrix, we introduce— for the first time—the low-rank assumption directly on the multi-label space, better reflecting intrinsic label structure; further, we develop a theoretically grounded bias-correction framework ensuring convergence. Contribution/Results: Extensive experiments on multiple real-world datasets demonstrate significant improvements in prediction accuracy and robustness to annotation noise, effectively mitigating performance degradation induced by label imperfections.

Technology Category

• Machine Learning: Multi-class/Multi-label Learning & Extreme Classification
• Computer Vision: Representation Learning for Vision
• Natural Language Processing: Safety and Robustness

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-label learning (MLL) has gained attention for its ability to represent real-world data. Label Distribution Learning (LDL), an extension of MLL to learning from label distributions, faces challenges in collecting accurate label distributions. To address the issue of biased annotations, based on the low-rank assumption, existing works recover true distributions from biased observations by exploring the label correlations. However, recent evidence shows that the label distribution tends to be full-rank, and naive apply of low-rank approximation on biased observation leads to inaccurate recovery and performance degradation. In this paper, we address the LDL with biased annotations problem from a novel perspective, where we first degenerate the soft label distribution into a hard multi-hot label and then recover the true label information for each instance. This idea stems from an insight that assigning hard multi-hot labels is often easier than assigning a soft label distribution, and it shows stronger immunity to noise disturbances, leading to smaller label bias. Moreover, assuming that the multi-label space for predicting label distributions is low-rank offers a more reasonable approach to capturing label correlations. Theoretical analysis and experiments confirm the effectiveness and robustness of our method on real-world datasets.

Problem

• Label Noise
• Multi-label Learning
• Label Distribution Learning

Innovation

• Hard Multi-Hot Labeling
• Low-Rank Multilabel Space
• Bias-Robust Learning