MOSS: Multi-Objective Optimization for Stable Rule Sets

2025-06-02

AI Summary

In interpretable rule learning, simultaneously achieving sparsity, accuracy, and stability remains challenging due to inherent trade-offs among these objectives. Method: This work is the first to explicitly incorporate stability as a core dimension in multi-objective optimization, proposing a joint optimization framework based on multi-objective integer programming. To efficiently compute the accuracy–stability Pareto frontier, we design a customized cutting-plane algorithm that overcomes scalability and efficiency limitations of commercial solvers. Contribution/Results: Our method significantly outperforms existing rule-ensemble approaches across multiple benchmark datasets. It maintains high predictive accuracy while substantially improving model stability—enabling users to flexibly balance accuracy and robustness. By unifying interpretability with reliability, the framework advances trustworthy, human-understandable machine learning.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization

Abstract

We present MOSS, a multi-objective optimization framework for constructing stable sets of decision rules. MOSS incorporates three important criteria for interpretability: sparsity, accuracy, and stability, into a single multi-objective optimization framework. Importantly, MOSS allows a practitioner to rapidly evaluate the trade-off between accuracy and stability in sparse rule sets in order to select an appropriate model. We develop a specialized cutting plane algorithm in our framework to rapidly compute the Pareto frontier between these two objectives, and our algorithm scales to problem instances beyond the capabilities of commercial optimization solvers. Our experiments show that MOSS outperforms state-of-the-art rule ensembles in terms of both predictive performance and stability.

Problem

• Optimizes sparsity, accuracy, stability in rule sets
• Evaluates trade-off between accuracy and stability
• Scales beyond commercial solvers for Pareto frontier

Innovation

• Multi-objective optimization for rule sets
• Cutting plane algorithm for Pareto frontier
• Balances sparsity, accuracy, and stability