Exact Instance Compression for Convex Empirical Risk Minimization via Color Refinement

2026-01-31

AI Summary

Although convex empirical risk minimization (ERM) enjoys strong theoretical guarantees, its large-scale optimization remains computationally expensive. This work proposes a lossless compression framework based on color refinement—a graph isomorphism technique—extending it for the first time from linear and quadratic programming to general differentiable convex ERM problems, thereby enabling exact and broadly applicable instance compression. By integrating insights from convex optimization theory, the framework yields efficient compression algorithms for a variety of models, including linear and polynomial regression, logistic regression, elastic net, and kernel methods. Empirical evaluations across multiple benchmark datasets demonstrate that the proposed approach substantially reduces computational overhead while rigorously preserving model performance.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Data Mining & Knowledge Management: Data Compression
• Computer Vision: Learning & Optimization for CV

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Empirical risk minimization (ERM) can be computationally expensive, with standard solvers scaling poorly even in the convex setting. We propose a novel lossless compression framework for convex ERM based on color refinement, extending prior work from linear programs and convex quadratic programs to a broad class of differentiable convex optimization problems. We develop concrete algorithms for a range of models, including linear and polynomial regression, binary and multiclass logistic regression, regression with elastic-net regularization, and kernel methods such as kernel ridge regression and kernel logistic regression. Numerical experiments on representative datasets demonstrate the effectiveness of the proposed approach.

Problem

• Empirical Risk Minimization
• Convex Optimization
• Computational Complexity
• Lossless Compression
• Color Refinement

Innovation

• color refinement
• lossless compression
• convex empirical risk minimization
• exact instance compression
• differentiable convex optimization