🤖 AI Summary
This work addresses the trade-off between computational cost and model stability in selecting the number of trees in random forests, noting that existing plateau-based tuning methods exhibit nondeterministic fluctuations even after apparent convergence. The study formulates ensemble size selection as a birth–death Markov chain on a geometric grid and derives its stationary distribution using local balance principles, revealing that plateau search is inherently a stochastic process. By introducing a symmetric correction update rule, the authors establish a theoretical framework for the stationary distribution of ternary plateau search, analyzed via a folded normal approximation centered at the optimum and a local Gaussian approximation. They prove that both the stationary center \(B_*\) and its standard deviation \(\sigma_{B,*}\) scale as \(O(\varepsilon^{-2})\), the variance as \(O(\varepsilon^{-4})\), and crucially, that the relative width of the distribution depends solely on the scaling factor and update rule, independent of the precision parameter \(\varepsilon\).
📝 Abstract
The number of trees is a central computational parameter in Random Forests: increasing it reduces finite-ensemble variability but increases training and prediction cost. Plateau-based tuning adapts this parameter through local comparisons of out-of-bag scores at a geometric triplet of tree counts. After the remaining hyperparameters have stabilized, however, the central triplet point need not converge to a deterministic value; instead, it fluctuates around a stationary regime.
This paper develops a stationary-distribution theory for this process. The central ensemble size $B_t$ is modeled as a birth-death Markov chain on a geometric grid, and its stationary distribution is derived through local balance. Under a leading centered folded-normal approximation, equilibrium equations are obtained for the original update rule and a symmetric modified variant, implying that the stationary center $B_*=O(\varepsilon^{-2})$ as $\varepsilon\downarrow 0$.
The stationary spread is also characterized. A local Gaussian approximation and a Fokker-Planck interpretation give grid-level variance constants. After conversion to the ensemble-size scale, $σ_{B,*}=O(\varepsilon^{-2})$, while the variance is $O(\varepsilon^{-4})$. The leading relative spread is independent of $\varepsilon$ and controlled by the scale factor and update rule. These results interpret plateau-based Random Forest tuning as a stochastic process rather than a deterministic stopping rule.