Between Resolution Collapse and Variance Inflation: Weighted Conformal Anomaly Detection in Low-Data Regimes

πŸ“… 2026-03-24
πŸ“ˆ Citations: 0
✨ Influential: 0
πŸ“„ PDF
πŸ€– AI Summary
This work addresses the challenge of conformal anomaly detection under data distribution shifts and limited sample regimes, where standard importance weighting drastically reduces the effective sample size, leading to overly conservative p-values or inflated variance that impairs anomaly identification. To overcome this, the authors propose a continuous inference relaxation framework that, for the first time, integrates continuous weighted kernel density estimation into conformal anomaly detection. By locally adapting to non-stationary data distributions, the method decouples the trade-off between tail resolution and stability while preserving marginal coverage guarantees. This approach mitigates the loss of statistical power induced by discretization, eliminates Monte Carlo variability, and substantially enhances detection capability in low-data settings, successfully recovering anomalies missed by discrete baseline methods.

Technology Category

Application Category

πŸ“ Abstract
Standard conformal anomaly detection provides marginal finite-sample guarantees under the assumption of exchangeability . However, real-world data often exhibit distribution shifts, necessitating a weighted conformal approach to adapt to local non-stationarity. We show that this adaptation induces a critical trade-off between the minimum attainable p-value and its stability. As importance weights localize to relevant calibration instances, the effective sample size decreases. This can render standard conformal p-values overly conservative for effective error control, while the smoothing technique used to mitigate this issue introduces conditional variance, potentially masking anomalies. We propose a continuous inference relaxation that resolves this dilemma by decoupling local adaptation from tail resolution via continuous weighted kernel density estimation. While relaxing finite-sample exactness to asymptotic validity, our method eliminates Monte Carlo variability and recovers the statistical power lost to discretization. Empirical evaluations confirm that our approach not only restores detection capabilities where discrete baselines yield zero discoveries, but outperforms standard methods in statistical power while maintaining valid marginal error control in practice.
Problem

Research questions and friction points this paper is trying to address.

conformal anomaly detection
distribution shift
importance weighting
low-data regimes
p-value stability
Innovation

Methods, ideas, or system contributions that make the work stand out.

conformal anomaly detection
importance weighting
continuous inference
kernel density estimation
low-data regime
πŸ”Ž Similar Papers
2024-05-29arXiv.orgCitations: 0
πŸ’Ό Related Jobs
No related jobs found.
O
Oliver HennhΓΆfer
Intelligent Systems Research Group (ISRG), Karlsruhe University of Applied Sciences, Germany
C
Christine Preisach
Intelligent Systems Research Group (ISRG), Karlsruhe University of Applied Sciences, Germany