Task Prototype-Based Knowledge Retrieval for Multi-Task Learning from Partially Annotated Data

2026-01-12

AI Summary

This work addresses the challenge of negative transfer in multi-task learning when only a subset of tasks has labeled data, a scenario where conventional approaches often suffer from unreliable task relationships. To mitigate this issue, the authors propose a knowledge retrieval framework grounded in task prototypes: task-specific characteristics are encoded via prototype embeddings to quantify inter-task affinities, and a knowledge retrieval Transformer adaptively refines feature representations based on these affinities. Furthermore, an Affinity-aware Knowledge Generation (AKG) loss is introduced to guide learning without relying on predictions from unlabeled tasks. Experiments demonstrate that the proposed method significantly improves performance under partial labeling conditions, effectively alleviating negative transfer and confirming its robustness and efficacy.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Multiagent Systems: Multiagent Learning
• Natural Language Processing: Sentence-level Semantics, Textual Inference, etc.

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Multi-task learning (MTL) is critical in real-world applications such as autonomous driving and robotics, enabling simultaneous handling of diverse tasks. However, obtaining fully annotated data for all tasks is impractical due to labeling costs. Existing methods for partially labeled MTL typically rely on predictions from unlabeled tasks, making it difficult to establish reliable task associations and potentially leading to negative transfer and suboptimal performance. To address these issues, we propose a prototype-based knowledge retrieval framework that achieves robust MTL instead of relying on predictions from unlabeled tasks. Our framework consists of two key components: (1) a task prototype embedding task-specific characteristics and quantifying task associations, and (2) a knowledge retrieval transformer that adaptively refines feature representations based on these associations. To achieve this, we introduce an association knowledge generating (AKG) loss to ensure the task prototype consistently captures task-specific characteristics. Extensive experiments demonstrate the effectiveness of our framework, highlighting its potential for robust multi-task learning, even when only a subset of tasks is annotated.

Problem

• multi-task learning
• partially annotated data
• task association
• negative transfer
• knowledge retrieval

Innovation

• prototype-based knowledge retrieval
• multi-task learning
• partially annotated data
• task association
• knowledge retrieval transformer