Exact characterization of {epsilon}-Safe Decision Regions for exponential family distributions and Multi Cost SVM approximation

2025-01-29

AI Summary

This paper addresses the challenge of ensuring prediction reliability under class imbalance in safety-critical applications by introducing a verifiable classification decision mechanism. Specifically, it tackles how to rigorously define a “safety-prediction region” wherein predictions for the target class are guaranteed to be highly reliable. Methodologically, it (1) formally defines the ε-safe decision region—the input subspace where the predicted probability for the target class is at least 1−ε; (2) derives a closed-form analytical solution for this region under the exponential family distribution assumption; and (3) proposes a novel multi-cost SVM algorithm to robustly and scalably approximate the region for non-exponential-family models and imbalanced settings. Theoretical analysis proves the region’s controllability and designability. Empirical evaluation on multiple imbalanced benchmarks demonstrates significant improvements in both safety-prediction coverage and confidence. All code and experimental artifacts are fully open-sourced to ensure reproducibility.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Natural Language Processing: Safety and Robustness
• Computer Vision: Adversarial Attacks & Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Probabilistic guarantees on the prediction of data-driven classifiers are necessary to define models that can be considered reliable. This is a key requirement for modern machine learning in which the goodness of a system is measured in terms of trustworthiness, clearly dividing what is safe from what is unsafe. The spirit of this paper is exactly in this direction. First, we introduce a formal definition of {epsilon}-Safe Decision Region, a subset of the input space in which the prediction of a target (safe) class is probabilistically guaranteed. Second, we prove that, when data come from exponential family distributions, the form of such a region is analytically determined and controllable by design parameters, i.e. the probability of sampling the target class and the confidence on the prediction. However, the request of having exponential data is not always possible. Inspired by this limitation, we developed Multi Cost SVM, an SVM based algorithm that approximates the safe region and is also able to handle unbalanced data. The research is complemented by experiments and code available for reproducibility.

Problem

• Machine Learning
• Safe Prediction Range
• Imbalanced Data

Innovation

• Exponential Family Distributions
• Multi-cost SVM Approximation
• ε-Safe Decision Region