Feature Importance Guided Random Forest Learning with Simulated Annealing Based Hyperparameter Tuning

2025-10-31

AI Summary

Traditional random forests suffer from insufficient feature utilization and limited generalization capability in credit risk assessment, IoT anomaly detection, early medical diagnosis, and high-dimensional bioinformatics analysis. To address these limitations, this paper proposes an importance-aware random forest optimization framework. Our method introduces two key innovations: (1) a feature-importance-guided probabilistic sampling mechanism to enhance the capture of discriminative signals, and (2) a coupled simulated annealing algorithm for adaptive global hyperparameter optimization. Crucially, the framework preserves model interpretability while substantially improving classification accuracy and cross-domain generalization performance. Extensive experiments on real-world datasets demonstrate that our approach consistently outperforms baseline models in prediction accuracy across diverse application domains. Moreover, it yields more robust and discriminative feature importance rankings, facilitating reliable downstream interpretation and decision support.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

This paper introduces a novel framework for enhancing Random Forest classifiers by integrating probabilistic feature sampling and hyperparameter tuning via Simulated Annealing. The proposed framework exhibits substantial advancements in predictive accuracy and generalization, adeptly tackling the multifaceted challenges of robust classification across diverse domains, including credit risk evaluation, anomaly detection in IoT ecosystems, early-stage medical diagnostics, and high-dimensional biological data analysis. To overcome the limitations of conventional Random Forests, we present an approach that places stronger emphasis on capturing the most relevant signals from data while enabling adaptive hyperparameter configuration. The model is guided towards features that contribute more meaningfully to classification and optimizing this with dynamic parameter tuning. The results demonstrate consistent accuracy improvements and meaningful insights into feature relevance, showcasing the efficacy of combining importance aware sampling and metaheuristic optimization.

Problem

• Enhancing Random Forest classifiers through feature importance guided learning
• Optimizing hyperparameters using simulated annealing for improved generalization
• Addressing robust classification challenges across diverse high-dimensional domains

Innovation

• Feature importance guided probabilistic sampling in Random Forest
• Simulated Annealing for adaptive hyperparameter tuning
• Dynamic parameter optimization with importance-aware feature selection