Efficient Large-Scale Learning of Minimax Risk Classifiers

2025-11-18

AI Summary

Training minimax risk classifiers (MRCs) for large-scale multi-class problems is computationally prohibitive; existing stochastic subgradient methods fail to effectively optimize the max-expected-loss objective. Method: We propose a deterministic optimization framework integrating constraint generation and column generation, eliminating stochastic approximations. It iteratively expands both the constraint set (samples) and the category subset (classes), enabling scalable optimization over large datasets and high-dimensional label spaces. Contribution/Results: This is the first deterministic algorithm supporting large-scale multi-class MRC training. On multiple benchmark datasets, it achieves 10–100× speedup over conventional methods—with acceleration increasing as the number of classes grows—while strictly preserving the MRC’s robustness guarantees and classification accuracy.

Technology Category

• Machine Learning: Optimization
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Supervised learning with large-scale data usually leads to complex optimization problems, especially for classification tasks with multiple classes. Stochastic subgradient methods can enable efficient learning with a large number of samples for classification techniques that minimize the average loss over the training samples. However, recent techniques, such as minimax risk classifiers (MRCs), minimize the maximum expected loss and are not amenable to stochastic subgradient methods. In this paper, we present a learning algorithm based on the combination of constraint and column generation that enables efficient learning of MRCs with large-scale data for classification tasks with multiple classes. Experiments on multiple benchmark datasets show that the proposed algorithm provides upto a 10x speedup for general large-scale data and around a 100x speedup with a sizeable number of classes.

Problem

• Efficient large-scale learning for minimax risk classifiers
• Overcoming limitations of stochastic subgradient methods
• Accelerating classification with multiple classes using constraint generation

Innovation

• Combines constraint and column generation for efficiency
• Enables large-scale minimax risk classifier learning
• Provides significant speedup for multi-class classification