Scalable Tree Ensemble Proximities in Python

2026-01-06

arXiv.org

AI Summary

This work addresses the scalability limitations of traditional tree ensemble methods for computing sample similarity, which suffer from quadratic time and memory complexity due to explicit pairwise comparisons. The authors propose a separable, weighted leaf collision similarity framework that formalizes a novel class of similarity measures amenable to exact sparse matrix factorization. By restricting computations to the leaf-node level and leveraging the inherent sparsity of tree-based representations, the method eliminates the need for explicit pairwise evaluations. Integrating decision tree structures with sparse linear algebra techniques, the approach enables efficient nearest-neighbor computation on standard CPUs, scaling to hundreds of thousands of samples while substantially reducing both runtime and memory consumption—thereby overcoming a fundamental bottleneck in the scalability of conventional tree-based similarity methods.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Sampling/Simulation-based Search
• Data Mining & Knowledge Management: Scalability, Parallel & Distributed Systems

Application Category

• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Tree ensemble methods such as Random Forests naturally induce supervised similarity measures through their decision tree structure, but existing implementations of proximities derived from tree ensembles typically suffer from quadratic time or memory complexity, limiting their scalability. In this work, we introduce a general framework for efficient proximity computation by defining a family of Separable Weighted Leaf-Collision Proximities. We show that any proximity measure in this family admits an exact sparse matrix factorization, restricting computation to leaf-level collisions and avoiding explicit pairwise comparisons. This formulation enables low-memory, scalable proximity computation using sparse linear algebra in Python. Empirical benchmarks demonstrate substantial runtime and memory improvements over traditional approaches, allowing tree ensemble proximities to scale efficiently to datasets with hundreds of thousands of samples on standard CPU hardware.

Problem

• tree ensemble
• proximity
• scalability
• quadratic complexity
• similarity measure

Innovation

• tree ensemble proximities
• scalable similarity
• sparse matrix factorization
• leaf-collision
• random forests