Unsupervised Ensemble Learning Through Deep Energy-based Models

2026-01-28

AI Summary

This work addresses the challenge of effectively aggregating predictions from multiple learners in the absence of ground-truth labels and additional data. We propose an unsupervised ensemble learning method based on deep energy models that constructs a high-performance meta-learner using only the outputs of base learners, without requiring labels, input features, or task-specific priors. To our knowledge, this is the first approach to integrate deep energy models into unsupervised ensemble learning, enabling the modeling of complex dependencies among learners while providing theoretical guarantees under the conditional independence assumption. Extensive experiments demonstrate that our method consistently outperforms existing approaches on both standard and custom ensemble datasets, with particularly strong performance in challenging scenarios such as mixture-of-experts settings, making it well-suited for applications where data scarcity or privacy constraints limit access to labeled information.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Relational Probabilistic Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Unsupervised ensemble learning emerged to address the challenge of combining multiple learners'predictions without access to ground truth labels or additional data. This paradigm is crucial in scenarios where evaluating individual classifier performance or understanding their strengths is challenging due to limited information. We propose a novel deep energy-based method for constructing an accurate meta-learner using only the predictions of individual learners, potentially capable of capturing complex dependence structures between them. Our approach requires no labeled data, learner features, or problem-specific information, and has theoretical guarantees for when learners are conditionally independent. We demonstrate superior performance across diverse ensemble scenarios, including challenging mixture of experts settings. Our experiments span standard ensemble datasets and curated datasets designed to test how the model fuses expertise from multiple sources. These results highlight the potential of unsupervised ensemble learning to harness collective intelligence, especially in data-scarce or privacy-sensitive environments.

Problem

• Unsupervised ensemble learning
• Energy-based models
• Meta-learning
• Label-free fusion
• Collective intelligence

Innovation

• unsupervised ensemble learning
• deep energy-based models
• meta-learner
• conditional independence
• collective intelligence