Importance Sampling for Event Discovery via Guesswork

2026-06-23

AI Summary

This work addresses the limitation of traditional importance sampling, which focuses on estimating rare-event probabilities, whereas many applications prioritize minimizing the time to hit a set of rare trajectories. The authors propose a new paradigm centered on guesswork from information theory, defining discovery quality as the description length—i.e., surprisal—of a trajectory under a nominal model. They formulate an optimization objective \( H(Q) + D(Q\|P) \) to design sampling strategies that blend stochastic exploration with systematic search. Under i.i.d. assumptions, large deviations theory is employed to analyze type-defined rare sets, showing that this objective not only characterizes discovery optimality but also serves as a lexicographic tiebreaker among strategies achieving identical hitting times under a fixed computational budget.

Abstract

Traditional importance sampling (IS) is designed to estimate rare-event probabilities by minimizing estimator variance. However, many applications prioritize rapid discovery: the generation of a trajectory within a rare set $A_n$. This requires a shift from ensemble-based estimation to a design principle focused on the hitting time $τ_{A_n} := \inf\{t \ge 1 : Y_t^n \in A_n\}$. We formalize a Quality of Discovery problem as the problem of minimizing the description length (surprisal) of the discovered trajectory under the nominal model $p$. We prove that minimizing this description length is equivalent to minimizing the nominal rank exponent $J_{\mathrm{rank}}(q_n) := \lim_{n\to\infty} \frac{1}{n} \log G_n(Y^n)$, where $G_n(x^n)$ is the guesswork of sequence $x^n$. For i.i.d.\ models and type-defined rare sets $Γ$, we show that while classical IS targets the mass-dominating type $Q_{\mathrm{IS}}^* \in \arg\min_{Q \in Γ} D(Q\|p)$, discovery optimality is achieved by $Q_{\mathrm{GW}}^* \in \arg\min_{Q \in Γ} [H(Q) + D(Q\|p)]$. This framework identifies a fundamental rule: minimizing the guesswork exponent ensures the discovered sequence is the "least surprising" representative of the set relative to the nominal model's search order. We further demonstrate that under budgetary constraints, this exponent serves as a lexicographic tie-breaker when the hitting-time minimizer is not unique. This establishes $H(Q) + D(Q\|p)$ as a natural objective for discovery-based importance sampling, providing a formal bridge between randomized sampling and systematic search.

Problem

• importance sampling
• rare-event discovery
• guesswork
• hitting time
• description length

Innovation

• importance sampling
• guesswork
• rare-event discovery
• description length
• hitting time