A Unified Algebraic Framework for Classification Performance Evaluation

2026-07-04

AI Summary

Existing classification evaluation methods lack a unified framework, limiting their generalizability across diverse scenarios such as binary, multiclass, multilabel, ordinal, hierarchical, cost-sensitive, and soft-label settings. This work proposes a unified algebraic evaluation framework that leverages binary indicator matrices of ground-truth and predicted labels, combined with three aggregation operators—global (micro), column-wise (macro/weighted), and row-wise (sample-averaged)—to automatically extend any TP/TN-based binary metric to arbitrary classification tasks. The framework further incorporates t-norms for soft labels, cumulative encoding for ordinal classification, and cost matrices to unify regression-like metrics. Theoretical analysis establishes key properties including the equivalence of micro and weighted macro aggregations, uniqueness of t-norms, skew invariance, and consistency between micro-F1 and accuracy. Extensive experiments validate the framework’s generality and correctness.

Abstract

We propose a unified algebraic framework for classification performance evaluation that encompasses binary, multiclass, multilabel, ordinal, hierarchical, cost-sensitive, and soft-label settings within a single formalism. The foundation is a representation of actual and predicted labels as binary indicator matrices, combined with three aggregation operators -- global, column-wise, and row-wise -- that correspond exactly to micro, macro/weighted, and exemplar averaging. Any binary performance measure expressed in terms of true/positive/negative counts extends automatically to all settings by substituting these operators, generating multiclass and multilabel versions without measure-specific derivations. The framework further accommodates soft classifier outputs via argmax or thresholding, soft ground truth via triangular norms, ordinal classification via membership functions or cumulative encodings, and cost-sensitive evaluation via a cost matrix that subsumes MAE and MSE as special cases. We establish several theoretical results: micro-averaging equals denominator-weighted macro-averaging; the product $t$-norm is the unique one preserving the confusion-matrix partition; skew-invariant measures are characterised as functions of recall and specificity; and micro-precision, micro-recall, and micro-$F_1$ are all equal to accuracy in multiclass settings. Empirical illustrations on synthetic and real data confirm the theoretical findings.

Problem

• classification evaluation
• performance metrics
• unified framework
• multilabel classification
• cost-sensitive learning

Innovation

• unified algebraic framework
• classification evaluation
• aggregation operators
• soft labels
• cost-sensitive learning