Rethinking Multi-Objective Learning through Goal-Conditioned Supervised Learning

2024-12-12

AI Summary

To address task conflict, poor generalization, and high computational complexity in multi-objective learning, this paper proposes a general framework based on Goal-Conditioned Supervised Learning (GCSL). The method decouples scalar objectives into interpretable multidimensional goal vectors and enables end-to-end joint optimization of multiple objectives directly from offline sequential data. It formally characterizes objective feasibility and introduces a novel goal generation mechanism that requires no specialized model architecture, explicit constraints, or complex optimization procedures. Experiments on real-world recommendation datasets demonstrate that the approach significantly improves multi-objective trade-off performance while maintaining high efficiency, scalability, and strong generalization across diverse tasks and domains.

Technology Category

• Machine Learning: Multimodal Learning
• Search and Optimization: Learning to Search
• Multiagent Systems: Multiagent Learning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Multi-objective learning aims to optimize multiple objectives simultaneously with a single model for achieving a balanced and satisfying performance on all these objectives. However, it suffers from the difficulty to formalize and conduct the exact learning process, especially considering the possible conflicts between objectives. Existing approaches explores to resolve this primarily in two directions: adapting modeling structure or constraining optimization with certain assumptions. However, a primary issue is that their presuppositions for the effectiveness of their design are insufficient to guarantee the its generality in real-world applications. What's worse, the high space and computation complexity issue makes it even harder to apply them in large-scale, complicated environment such as the recommender systems. To address these issues, we propose a general framework for automatically learning to achieve multiple objectives based on the existing sequential data. We apply the goal-conditioned supervised learning (GCSL) framework to multi-objective learning, by extending the definition of goals from one-dimensional scalar to multi-dimensional vector that perfectly disentangle the representation of different objectives. Meanwhile, GCSL enables the model to simultaneously learn to achieve each objective in a concise supervised learning way, simply guided by existing sequences in the offline data. No additional constraint, special model structure design, or complex optimization algorithms are further required. Apart from that, we formally analyze the property of the goals in GCSL and then firstly propose a goal-generation framework to gain achievable and reasonable goals for inference. Extensive experiments are conducted on real-world recommendation datasets, demonstrating the effectiveness of the proposed method and exploring the feasibility of the goal-generation strategies in GCSL.

Problem

• Multi-task Learning
• Task Conflict Resolution
• Recommendation Systems

Innovation

• GCSL Framework
• Multi-task Learning
• Goal Generation Strategy