Global Sequential Testing for Multi-Stream Auditing

2026-02-24

AI Summary

This work addresses the challenge of efficient continual auditing for machine learning systems operating under multiple data streams by formulating it as a global sequential hypothesis testing problem. The authors propose a balanced sequential testing method based on test martingale fusion, which retains the error control guarantees of Bonferroni correction under sparse alternatives while significantly reducing the expected stopping time to $O((1/k)\ln(1/\alpha))$ in dense settings. By integrating sequential testing with multiple comparison correction, the proposed approach demonstrates superior detection efficiency and faster stopping performance on both synthetic and real-world datasets.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Data Mining & Knowledge Management: Anomaly/Outlier Detection
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Across many risk-sensitive areas, it is critical to continuously audit the performance of machine learning systems and detect any unusual behavior quickly. This can be modeled as a sequential hypothesis testing problem with $k$ incoming streams of data and a global null hypothesis that asserts that the system is working as expected across all $k$ streams. The standard global test employs a Bonferroni correction and has an expected stopping time bound of $O\left(\ln\frac{k}α\right)$ when $k$ is large and the significance level of the test, $α$, is small. In this work, we construct new sequential tests by using ideas of merging test martingales with different trade-offs in expected stopping times under different, sparse or dense alternative hypotheses. We further derive a new, balanced test that achieves an improved expected stopping time bound that matches Bonferroni's in the sparse setting but that naturally results in $O\left(\frac{1}{k}\ln\frac{1}α\right)$ under a dense alternative. We empirically demonstrate the effectiveness of our proposed tests on synthetic and real-world data.

Problem

• sequential testing
• multi-stream auditing
• global hypothesis testing
• anomaly detection
• machine learning monitoring

Innovation

• sequential testing
• test martingales
• multi-stream auditing
• Bonferroni correction
• sparse vs dense alternatives