PEARL: Performance-Enhanced Aggregated Representation Learning

2025-09-29

AI Summary

Single-representation learning methods often overlook task-critical features, limiting downstream performance. Method: This paper proposes a general and flexible multi-representation aggregation framework that estimates representation weights via proxy loss functions. Theoretically, it guarantees asymptotically optimal risk and adaptively assigns nonzero weights to correctly specified models, balancing predictive optimality with model selection capability. The framework accommodates diverse standard loss functions, ensuring broad applicability. Contribution/Results: Extensive experiments across multiple downstream tasks demonstrate consistent superiority over state-of-the-art baselines, yielding significant improvements in prediction accuracy. The method exhibits strong robustness and wide applicability across heterogeneous domains and model architectures.

Technology Category

• Machine Learning: Representation Learning
• Computer Vision: Representation Learning for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Representation learning is a key technique in modern machine learning that enables models to identify meaningful patterns in complex data. However, different methods tend to extract distinct aspects of the data, and relying on a single approach may overlook important insights relevant to downstream tasks. This paper proposes a performance-enhanced aggregated representation learning method, which combines multiple representation learning approaches to improve the performance of downstream tasks. The framework is designed to be general and flexible, accommodating a wide range of loss functions commonly used in machine learning models. To ensure computational efficiency, we use surrogate loss functions to facilitate practical weight estimation. Theoretically, we prove that our method asymptotically achieves optimal performance in downstream tasks, meaning that the risk of our predictor is asymptotically equivalent to the theoretical minimum. Additionally, we derive that our method asymptotically assigns nonzero weights to correctly specified models. We evaluate our method on diverse tasks by comparing it with advanced machine learning models. The experimental results demonstrate that our method consistently outperforms baseline methods, showing its effectiveness and broad applicability in real-world machine learning scenarios.

Problem

• Combining multiple representation learning approaches to capture diverse data insights
• Enhancing downstream task performance through aggregated representation learning
• Achieving asymptotic optimal performance with flexible loss function accommodation

Innovation

• Combines multiple representation learning approaches
• Uses surrogate loss functions for computational efficiency
• Asymptotically achieves optimal performance in tasks