Pareto Low-Rank Adapters: Efficient Multi-Task Learning with Preferences

2024-07-10

arXiv.org

AI Summary

This work addresses the challenge of preference-driven Pareto front modeling in multi-task learning, where existing Pareto Front Learning (PFL) methods suffer from poor scalability, slow convergence, high memory overhead, and inconsistency between preference vectors and objective-space mappings. We propose PaLoRA—a parameter-efficient and scalable method that introduces the first convex-hull-based continuous parameterization of the Pareto front via low-rank adaptation (LoRA), enabling explicit, single-model representation of the entire frontier. Additionally, we design a deterministic preference vector sampling strategy to jointly optimize shared and task-specific feature representations. Evaluated on multi-task benchmarks including scene understanding, PaLoRA achieves significant improvements over state-of-the-art methods, scales effectively to large models, reduces memory consumption by 23.8–31.7×, and simultaneously accelerates convergence and enhances preference–objective mapping consistency.

Technology Category

• Machine Learning: Learning Preferences or Rankings
• Search and Optimization: Learning to Search
• Knowledge Representation and Reasoning: Preferences

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Multi-task trade-offs in machine learning can be addressed via Pareto Front Learning (PFL) methods that parameterize the Pareto Front (PF) with a single model. PFL permits to select the desired operational point during inference, contrary to traditional Multi-Task Learning (MTL) that optimizes for a single trade-off decided prior to training. However, recent PFL methodologies suffer from limited scalability, slow convergence, and excessive memory requirements, while exhibiting inconsistent mappings from preference to objective space. We introduce PaLoRA, a novel parameter-efficient method that addresses these limitations in two ways. First, we augment any neural network architecture with task-specific low-rank adapters and continuously parameterize the PF in their convex hull. Our approach steers the original model and the adapters towards learning general and task-specific features, respectively. Second, we propose a deterministic sampling schedule of preference vectors that reinforces this division of labor, enabling faster convergence and strengthening the validity of the mapping from preference to objective space throughout training. Our experiments show that PaLoRA outperforms state-of-the-art MTL and PFL baselines across various datasets, scales to large networks, reducing the memory overhead $23.8-31.7$ times compared with competing PFL baselines in scene understanding benchmarks.

Problem

• Efficient multi-task learning
• Scalable Pareto Front Learning
• Reduced memory overhead

Innovation

• Low-rank adapters enhance efficiency
• Deterministic preference sampling improves convergence
• Reduced memory overhead in large networks