Statistical few-shot learning for large-scale classification via parameter pooling

2025-04-15

AI Summary

In large-scale few-shot classification, high dimensionality, numerous classes, and extremely limited per-class samples lead to inaccurate covariance estimation and degraded performance of Linear Discriminant Analysis (LDA) and Quadratic Discriminant Analysis (QDA). To address this, we propose a parameter pooling method based on clustering sample covariance matrices. Our approach abandons LDA’s homoscedasticity assumption and employs model-driven spectral regularization for clustering, enabling unified modeling of both singular and non-singular covariances while establishing provable statistical estimation properties. Extensive experiments on synthetic and real-world datasets demonstrate substantial improvements in classification accuracy—particularly under challenging regimes with many classes, very low sample sizes per class, and high dimensionality—outperforming LDA, QDA, and other baselines in robustness and overall performance. The key innovation lies in the first integration of covariance matrix clustering with parameter pooling, jointly ensuring discriminative power, numerical stability, and theoretical interpretability.

Technology Category

• Machine Learning: Learning with Manifolds
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Mixed Discrete/Continuous Search

Application Category

• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

In large-scale few-shot learning for classification problems, often there are a large number of classes and few high-dimensional observations per class. Previous model-based methods, such as Fisher's linear discriminant analysis (LDA), require the strong assumptions of a shared covariance matrix between all classes. Quadratic discriminant analysis will often lead to singular or unstable covariance matrix estimates. Both of these methods can lead to lower-than-desired classification performance. We introduce a novel, model-based clustering method that can relax the shared covariance assumptions of LDA by clustering sample covariance matrices, either singular or non-singular. In addition, we study the statistical properties of parameter estimates. This will lead to covariance matrix estimates which are pooled within each cluster of classes. We show, using simulated and real data, that our classification method tends to yield better discrimination compared to other methods.

Problem

• Addresses large-scale few-shot learning with many classes
• Relaxes shared covariance assumptions in LDA via clustering
• Improves classification performance with pooled covariance estimates

Innovation

• Clusters sample covariance matrices for pooling
• Relaxes shared covariance assumptions of LDA
• Improves classification via parameter pooling