Safe cross-entropy-based importance sampling for rare event simulations

2025-09-08

AI Summary

For rare-event simulation with extremely low failure probabilities, conventional improved cross-entropy (ICE) methods suffer from slow convergence, poor stability, and reliance on pre-specified numbers of mixture components due to their exclusive use of light-tailed mixture models. To address these limitations, this paper proposes a robust cross-entropy-based importance sampling method. It employs a two-component mixture distribution—comprising both light-tailed and heavy-tailed components—to enhance tail-region exploration. A weighted cross-entropy penalized EM algorithm is designed to automatically prune redundant components, eliminating the need for manual specification of the mixture count. Parameter optimization is guided by Kullback–Leibler divergence minimization, substantially improving convergence speed, estimation accuracy, and robustness in rare-event simulation. Extensive benchmark evaluations demonstrate that the proposed method achieves faster convergence and lower estimation error than standard ICE, while adaptively identifying the optimal mixture structure.

Technology Category

• Machine Learning: Mixture of Experts (MoE)
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

The Improved Cross-Entropy (ICE) method is a powerful tool for estimating failure probabilities in reliability analysis. Its core idea is to approximate the optimal importance-sampling density by minimizing the forward Kullback-Leibler divergence within a chosen parametric family-typically a mixture model. However, conventional mixtures are often light-tailed, which leads to slow convergence and instability when targeting very small failure probabilities. Moreover, selecting the number of mixture components in advance can be difficult and may undermine stability. To overcome these challenges, we adopt a weighted cross-entropy-penalized expectation-maximization (EM) algorithm that automatically prunes redundant components during the iterative process, making the approach more stable. Furthermore, we introduce a novel two-component mixture that pairs a light-tailed distribution with a heavy-tailed one, enabling more effective exploration of the tail region and thus accelerating convergence for extremely small failure probabilities. We call the resulting method Safe-ICE and assess it on a variety of test problems. Numerical results show that Safe-ICE not only converges more rapidly and yields more accurate failure-probability estimates than standard ICE, but also identifies the appropriate number of mixture components without manual tuning.

Problem

• Improving stability in rare event probability estimation
• Addressing light-tailed mixture limitations for small probabilities
• Automating mixture component selection without manual tuning

Innovation

• Weighted cross-entropy-penalized EM algorithm
• Two-component light and heavy-tailed mixture
• Automatic pruning of redundant mixture components