A scalable version of MADD for big-data classification

2026-07-09

AI Summary

This work addresses the scalability bottleneck of traditional MADD (Median Absolute Deviation Distance) classifiers, whose computational complexity grows quadratically with sample size, rendering them impractical for high-dimensional large-scale data. To overcome this limitation, the authors propose a scalable MADD classifier that integrates representative sample selection with random Fourier feature approximation, enabling efficient application of MADD to massive datasets for the first time. The method preserves the strong discriminative power of MADD in high-dimensional spaces while substantially reducing computational overhead. Theoretical analysis and empirical evaluations demonstrate that the proposed approach achieves classification performance comparable to the original MADD with only a minimal increase in computational time, effectively balancing accuracy and efficiency.

Abstract

Distance-based classifiers are very popular, and the Euclidean distance is one of the most commonly used metrics in distance-based classifiers. However, classifiers based on the Euclidean distance often suffer in high-dimensional setups due to issues such as distance concentration, violation of neighborhood structures, and the presence of hubs. In high-dimension, low-sample-size (HDLSS) situations, a data-driven semi-metric called the Mean Absolute Difference of Distances (MADD) is known to circumvent these issues. But one major problem with MADD is that its computational complexity increases quadratically with the training sample size. As a result, the application of MADD becomes computationally challenging for big datasets that have both a high dimension as well as a large number of observations. In this paper, we propose a scalable version of MADD that significantly reduces its computational complexity while retaining its advantages. This speed-up is achieved by selecting a representative set during the computation of MADD. Further speed-ups are achieved by using the idea of Random Fourier Features, particularly when the sample size is very large. We establish that our proposed methods achieve performances similar to MADD but only at a fraction of its computing time, both theoretically as well as numerically. Our approach broadens the scope of MADD, allowing its use to big-data with a very large number of observations.

Problem

• MADD
• big-data classification
• high-dimensional data
• computational complexity
• distance concentration

Innovation

• MADD
• scalable classification
• high-dimensional data
• Random Fourier Features
• distance concentration