FUTURE: Flexible Unlearning for Tree Ensemble

2025-08-28

AI Summary

Existing sample unlearning methods for tree ensemble models suffer from poor generalizability, reliance on discrete tree structures, and low efficiency in large-scale settings. To address these limitations, this paper proposes FUTURE—a novel framework that reformulates unlearning as a differentiable gradient-based optimization problem. By introducing continuous relaxation and probabilistic model approximation, FUTURE enables end-to-end training without explicit dependence on tree topology. The approach supports flexible batch-wise unlearning, ensuring both universality across diverse ensemble architectures and scalability to massive datasets. Extensive experiments on multiple real-world benchmarks demonstrate that FUTURE significantly outperforms state-of-the-art methods in both unlearning accuracy and computational efficiency—particularly under complex ensemble configurations and large-scale data regimes.

Technology Category

• Machine Learning: Ensemble Methods
• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Tree ensembles are widely recognized for their effectiveness in classification tasks, achieving state-of-the-art performance across diverse domains, including bioinformatics, finance, and medical diagnosis. With increasing emphasis on data privacy and the extit{right to be forgotten}, several unlearning algorithms have been proposed to enable tree ensembles to forget sensitive information. However, existing methods are often tailored to a particular model or rely on the discrete tree structure, making them difficult to generalize to complex ensembles and inefficient for large-scale datasets. To address these limitations, we propose FUTURE, a novel unlearning algorithm for tree ensembles. Specifically, we formulate the problem of forgetting samples as a gradient-based optimization task. In order to accommodate non-differentiability of tree ensembles, we adopt the probabilistic model approximations within the optimization framework. This enables end-to-end unlearning in an effective and efficient manner. Extensive experiments on real-world datasets show that FUTURE yields significant and successful unlearning performance.

Problem

• Enables efficient unlearning in tree ensembles
• Addresses non-differentiability through probabilistic approximations
• Provides scalable forgetting for large datasets

Innovation

• Gradient-based optimization for unlearning
• Probabilistic model approximations framework
• End-to-end efficient forgetting method