Quantitative Group Testing and Pooled Data in the Linear Regime with Sublinear Tests

2024-08-01

arXiv.org

AI Summary

This paper investigates non-adaptive linear quantitative group testing (QGT) and its multi-class generalization—mixed-data detection—with the goal of accurately estimating class-wise item counts under linear-density scaling (i.e., fixed class proportions), using significantly fewer pooled tests than the total number of items (n). We propose a spatially coupled Bernoulli measurement matrix, paired with an approximate message passing (AMP) algorithm, enabling rigorous asymptotic performance analysis in both noiseless and noisy settings. Our approach achieves, for the first time in the linear-density regime, sublinear test complexity (o(n)) while guaranteeing almost-exact recovery. We establish theoretical robustness against measurement noise and finite-sample perturbations. Numerical simulations confirm substantial gains over i.i.d. Bernoulli designs and convex optimization baselines, particularly in moderate-dimensional regimes.

Technology Category

• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Calibration & Uncertainty Quantification
• Constraint Satisfaction and Optimization: Mixed Discrete/Continuous Optimization

Application Category

• Security and Privacy: Large-scale security measurements
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

In the pooled data problem, the goal is to identify the categories associated with a large collection of items via a sequence of pooled tests. Each pooled test reveals the number of items in the pool belonging to each category. A prominent special case is quantitative group testing (QGT), which is the case of pooled data with two categories. We consider these problems in the non-adaptive and linear regime, where the fraction of items in each category is of constant order. We propose a scheme with a spatially coupled Bernoulli test matrix and an efficient approximate message passing (AMP) algorithm for recovery. We rigorously characterize its asymptotic performance in both the noiseless and noisy settings, and prove that in the noiseless case, the AMP algorithm achieves almost-exact recovery with a number of tests sublinear in the number of items. For both QGT and pooled data, this is the first efficient scheme that provably achieves recovery in the linear regime with a sublinear number of tests, with performance degrading gracefully in the presence of noise. Numerical simulations illustrate the benefits of the spatially coupled scheme at finite dimensions, showing that it outperforms i.i.d. test designs as well as other recovery algorithms based on convex programming.

Problem

• Identifying item categories via pooled tests efficiently
• Achieving sublinear tests for quantitative group testing
• Robust recovery in noiseless and noisy settings

Innovation

• Spatially coupled Bernoulli test matrix design
• Efficient approximate message passing algorithm
• Sublinear tests with graceful noise degradation