Jump Like A Squirrel: Optimized Execution Step Order for Anytime Random Forest Inference

2026-03-02

AI Summary

This work addresses the challenge of maintaining high prediction accuracy in random forest inference under time interruptions on resource-constrained systems, where execution may be halted before all trees complete. To enable effective anytime behavior, we reformulate random forests as fine-grained anytime algorithms by treating individual internal nodes of decision trees as scheduling units and optimizing their execution order to maximize average accuracy under interruption. We introduce an exponential-time algorithm to find the optimal node ordering and propose two efficient polynomial-time heuristics—Forward and Backward Squirrel Order. Experimental results demonstrate that Backward Squirrel Order achieves 94% of the optimal performance and outperforms existing strategies by approximately 99%, significantly enhancing the robustness and accuracy of anytime random forest inference.

Technology Category

• Planning, Routing, and Scheduling: Scheduling under Uncertainty
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Heuristic Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Due to their efficiency and small size, decision trees and random forests are popular machine learning models used for classification on resource-constrained systems. In such systems, the available execution time for inference in a random forest might not be sufficient for a complete model execution. Ideally, the already gained prediction confidence should be retained. An anytime algorithm is designed to be able to be aborted anytime, while giving a result with an increasing quality over time. Previous approaches have realized random forests as anytime algorithms on the granularity of trees, stopping after some but not all trees of a forest have been executed. However, due to the way decision trees subdivide the sample space in every step, an increase in prediction quality is achieved with every additional step in one tree. In this paper, we realize decision trees and random forest as anytime algorithms on the granularity of single steps in trees. This approach opens a design space to define the step order in a forest, which has the potential to optimize the mean accuracy. We propose the Optimal Order, which finds a step order with a maximal mean accuracy in exponential runtime and the polynomial runtime heuristics Forward Squirrel Order and Backward Squirrel Order, which greedily maximize the accuracy for each additional step taken down and up the trees, respectively. Our evaluation shows, that the Backward Squirrel Order performs $\sim94\%$ as well as the Optimal Order and $\sim99\%$ as well as all other step orders.

Problem

• anytime algorithm
• random forest
• inference
• resource-constrained systems
• prediction confidence

Innovation

• anytime inference
• random forest
• step-level granularity
• execution order optimization
• squirrel order