BranchNet: A Neuro-Symbolic Learning Framework for Structured Multi-Class Classification

2025-07-02

AI Summary

This work addresses the challenge of jointly achieving interpretability and end-to-end optimization in structured multiclass classification tasks, where decision tree ensembles traditionally lack differentiability. We propose BranchNet: a novel method that automatically converts tree ensembles into sparse, partially connected neural networks. Its core innovation lies in the first neural-symbolic mapping of each decision path to a single hidden neuron—explicitly preserving the branching logic of trees while enabling gradient-based backpropagation and joint optimization. BranchNet requires no manual architecture design or hyperparameter tuning, naturally yielding compact, interpretable, and sparse models. Evaluated on multiple structured multiclass benchmarks, BranchNet significantly outperforms XGBoost (p < 0.01), with particularly pronounced gains on high-complexity tasks. It thus unifies three key desiderata: model interpretability, architectural compactness, and full end-to-end trainability.

Technology Category

• Machine Learning: Ensemble Methods
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Bridging structured and unstructured data

Abstract

We introduce BranchNet, a neuro-symbolic learning framework that transforms decision tree ensembles into sparse, partially connected neural networks. Each branch, defined as a decision path from root to a parent of leaves, is mapped to a hidden neuron, preserving symbolic structure while enabling gradient-based optimization. The resulting models are compact, interpretable, and require no manual architecture tuning. Evaluated on a suite of structured multi-class classification benchmarks, BranchNet consistently outperforms XGBoost in accuracy, with statistically significant gains. We detail the architecture, training procedure, and sparsity dynamics, and discuss the model's strengths in symbolic interpretability as well as its current limitations, particularly on binary tasks where further adaptive calibration may be beneficial.

Problem

• Transforms decision trees into sparse neural networks
• Enables gradient-based optimization with symbolic interpretability
• Outperforms XGBoost in structured multi-class classification

Innovation

• Transforms decision trees into sparse neural networks
• Preserves symbolic structure with gradient optimization
• Compact, interpretable models without manual tuning