Discriminative Subspace Emersion from learning feature relevances across different populations

2025-03-31

AI Summary

Addressing the challenges of high inter-class overlap, high-dimensional features, and scarce labeled data in cross-population binary classification, this paper proposes a general relevance learning framework that, for the first time, generalizes subspace learning into a discriminative subspace discovery method adaptable to multiple label sets. The method jointly models a shared discriminative subspace and decoupled basis learners via optimization-driven relevance estimation, with theoretical guarantees ensuring accurate extraction of cross-population common discriminative subspaces even under strong class overlap. Unlike conventional fixed-basis assumptions, its basis-agnostic adaptation mechanism significantly enhances robustness and generalization. Experiments on synthetic and real-world datasets demonstrate stable and precise subspace identification performance even under extremely high class overlap, establishing a novel paradigm for few-shot cross-population discriminative modeling.

Technology Category

• Machine Learning: Learning with Manifolds
• Search and Optimization: Mixed Discrete/Continuous Search
• Computer Vision: Representation Learning for Vision

Application Category

• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

In a given classification task, the accuracy of the learner is often hampered by finiteness of the training set, high-dimensionality of the feature space and severe overlap between classes. In the context of interpretable learners, with (piecewise) linear separation boundaries, these issues can be mitigated by careful construction of optimization procedures and/or estimation of relevant features for the task. However, when the task is shared across two disjoint populations the main interest is shifted towards estimating a set of features that discriminate the most between the two, when performing classification. We propose a new Discriminative Subspace Emersion (DSE) method to extend subspace learning toward a general relevance learning framework. DSE allows us to identify the most relevant features in distinguishing the classification task across two populations, even in cases of high overlap between classes. The proposed methodology is designed to work with multiple sets of labels and is derived in principle without being tied to a specific choice of base learner. Theoretical and empirical investigations over synthetic and real-world datasets indicate that DSE accurately identifies a common subspace for the classification across different populations. This is shown to be true for a surprisingly high degree of overlap between classes.

Problem

• Identify discriminative features across two populations
• Handle high-dimensional feature spaces and class overlap
• Learn common subspace for classification tasks

Innovation

• Discriminative Subspace Emersion for relevance learning
• Identifies relevant features across populations
• Works with multiple label sets generically