SNAP-UQ: Self-supervised Next-Activation Prediction for Single-Pass Uncertainty in TinyML

2025-08-18

AI Summary

Traditional uncertainty quantification (UQ) methods are computationally prohibitive for resource-constrained TinyML devices. Method: We propose SNAP-UQ—a lightweight, label-free, single-forward-pass UQ framework that leverages self-supervised predictive relationships between activations of adjacent deep layers to extract surprisal signals from quantized activations, mapping them directly to uncertainty scores via a compact monotonic mapper—eliminating auxiliary branches and multiple inferences. It integrates an int8-quantized head network and a single-path routing mechanism to minimize overhead. Results: On vision and audio tasks, SNAP-UQ achieves ≈0.9 AUROC for failure detection, reduces model size by 40–60%, and cuts inference latency by 25–35%, outperforming early-exit and deep ensemble baselines across all metrics.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Reasoning under Uncertainty: Uncertainty Representations
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We introduce extbf{SNAP-UQ}, a single-pass, label-free uncertainty method for TinyML that estimates risk from emph{depth-wise next-activation prediction}: tiny int8 heads forecast the statistics of the next layer from a compressed view of the previous one, and a lightweight monotone mapper turns the resulting surprisal into an actionable score. The design requires no temporal buffers, auxiliary exits, or repeated forward passes, and adds only a few tens of kilobytes to MCU deployments. Across vision and audio backbones, SNAP-UQ consistently reduces flash and latency relative to early-exit and deep ensembles (typically $sim$40--60% smaller and $sim$25--35% faster), with competing methods of similar accuracy often exceeding memory limits. In corrupted streams it improves accuracy-drop detection by several AUPRC points and maintains strong failure detection (AUROC $approx$0.9) in a single pass. Grounding uncertainty in layer-to-layer dynamics yields a practical, resource-efficient basis for on-device monitoring in TinyML.

Problem

• Estimates uncertainty in TinyML without labels or buffers
• Reduces memory and latency versus early-exit ensembles
• Improves corruption detection in single-pass deployments

Innovation

• Single-pass uncertainty method for TinyML
• Depth-wise next-activation prediction
• Lightweight monotone mapper for risk scores