Learning Sparse Decision Trees via Transformer Variational Auto-Encoders

2026-09-01

AI Summary

本文提出TREVIS方法，通过树变换变分自编码器探索潜在空间，以同时优化决策树的预测性能和结构稀疏性。

Abstract

Decision trees are among the most widely used models in machine learning, largely due to their transparent decision logic, making them well-suited for high-stakes decision-making contexts. However, most existing learning algorithms focus on predictive performance, overlooking the joint optimization of other desirable properties, such as structural sparsity. In this work we propose TREVIS, an approach for learning decision trees with respect to complex objectives, based on the exploration of the latent space of a Tree Transformer Variational Auto-Encoder (TTVAE). By mapping decision trees onto latent representations, TREVIS replaces the discrete search space with a continuous one, enabling gradient-based optimization via a differentiable surrogate model. We experiment with TREVIS for learning decision trees that jointly optimize predictive performance and sparsity. Results show that TREVIS discovers decision trees matching the predictive performance of existing near-optimal algorithms while improving their structural sparsity.

Problem

• Decision Trees
• Predictive Performance
• Structural Sparsity

Innovation

• TREVIS
• Tree Transformer Variational Auto-Encoder
• Structural Sparsity
• Gradient-based Optimization