Fast, Parallel, Query-Efficient Binary Classification

2026-07-04

AI Summary

This work addresses the problem of efficiently computing a maximum-margin separating hyperplane for binary classification on labeled datasets, with a focus on reducing matrix-vector query complexity, sequential runtime, and parallel depth. To this end, the authors propose two randomized optimization algorithms. The first achieves a total work complexity of Õ(γ⁻²/³·nnz(Φ) + γ⁻²(ω+1)/3) and parallel depth Õ(γ⁻²/³). The second algorithm further improves the work bound to Õ(γ⁻²/³·nnz(Φ) + γ⁻²) while maintaining Õ(γ⁻²/³) matrix-vector queries. Both methods surpass the efficiency limitations of deterministic approaches, offering significantly enhanced sequential and parallel performance while preserving near-optimal query complexity.

Abstract

We study the fundamental classification problem of computing a separating hyperplane for a binary-labeled dataset of size $n$ with normalized $d$-dimensional features. Letting $Φ\in \mathbb{R}^{n \times d}$ denote the feature matrix and $γ$ the margin of the maximum-margin separating hyperplane, we present a randomized algorithm that solves this problem in $\tilde{O}(γ^{-2/3}\, \operatorname{nnz}(Φ) + γ^{-2(ω+1)/3})$-sequential running time (work), $\tilde{O}(γ^{-2/3})$-parallel (computational) depth, and accesses $Φ$ only through $\tilde{O}(γ^{-2/3})$-matrix-vector queries (matvecs). We also present a second, faster randomized algorithm with a $\tilde{O}(γ^{-2/3}\, \operatorname{nnz}(Φ) + γ^{-2})$-sequential running time that uses $\tilde{O}(γ^{-2/3})$-matvecs to $Φ$, but achieves only $\tilde{O}(γ^{-4/3})$-parallel depth. Both algorithms match the near-optimal deterministic matvec complexity recently established by Kornowski and Shamir [2025], Karmarkar et al. [2026] and achieve improved sequential runtime and parallel depth, albeit at the expense of using randomness.

Problem

• binary classification
• separating hyperplane
• margin
• matrix-vector queries
• parallel depth

Innovation

• query-efficient
• parallel depth
• randomized algorithm
• maximum-margin hyperplane
• matrix-vector queries