Can Optimization Trajectories Explain Multi-Task Transfer?

2024-08-26

arXiv.org

AI Summary

This work investigates the mechanisms underlying generalization degradation in multi-task learning (MTL), specifically why mainstream optimization algorithms often fail to improve generalization. Through controlled experiments on deep neural networks, quantitative measurement of gradient conflicts, visualization of optimization trajectories, and attribution analysis of generalization error, we empirically establish— for the first time—that a substantial generalization gap emerges early in multi-task training, and crucially, this gap is decoupled from the degree of gradient conflict: while gradient conflicts shape task-specific optimization dynamics, they do not predict generalization performance. This finding challenges the prevailing MTL design paradigm that assumes mitigating gradient conflict is a necessary condition for improving generalization. Our primary contributions are: (i) identifying the early onset and independence of MTL generalization failure; (ii) refuting gradient conflict as a valid proxy for generalization; and (iii) providing new theoretical foundations for understanding task synergy and developing robust multi-task optimizers.

Technology Category

• Machine Learning: Optimization
• Search and Optimization: Learning to Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI

Abstract

Despite the widespread adoption of multi-task training in deep learning, little is understood about how multi-task learning (MTL) affects generalization. Prior work has conjectured that the negative effects of MTL are due to optimization challenges that arise during training, and many optimization methods have been proposed to improve multi-task performance. However, recent work has shown that these methods fail to consistently improve multi-task generalization. In this work, we seek to improve our understanding of these failures by empirically studying how MTL impacts the optimization of tasks, and whether this impact can explain the effects of MTL on generalization. We show that MTL results in a generalization gap (a gap in generalization at comparable training loss) between single-task and multi-task trajectories early into training. However, we find that factors of the optimization trajectory previously proposed to explain generalization gaps in single-task settings cannot explain the generalization gaps between single-task and multi-task models. Moreover, we show that the amount of gradient conflict between tasks is correlated with negative effects to task optimization, but is not predictive of generalization. Our work sheds light on the underlying causes for failures in MTL and, importantly, raises questions about the role of general purpose multi-task optimization algorithms.

Problem

• Multi-task Learning
• Skill Transfer
• Optimization Algorithms

Innovation

• Multitask Learning
• Contradictory Early-stage Performance
• Conflicting Task Impact