🤖 AI Summary
This work addresses the challenge of achieving bit-level exact verification of AI inference, which is hindered by the non-determinism of GPU floating-point operations and forces existing auditing approaches to rely on approximate matching—rendering them vulnerable to stealthy attacks. The authors uncover the mechanisms by which modern inference engines (e.g., vLLM, Hugging Face Transformers) produce deterministic yet non-invariant outputs under default configurations. They propose a software-level recomputation method that enables bit-accurate verification across different NVIDIA GPU variants without requiring identical hardware or incurring performance overhead. By modeling floating-point behavior, reconstructing internal engine states, and analyzing deterministic execution paths, the approach transforms accumulated rounding errors into auditable software-hardware fingerprints, establishing a tamper-resistant and verifiable foundation for AI governance.
📝 Abstract
Verifying claims about AI workloads is a pre- requisite for credible AI governance of covert adversaries (who comply with monitoring only when detection likelihood is high), yet the ap- parent non-determinism of GPU floating-point arithmetic forces auditors to accept approximate output matches. Covert adversaries can exploit un- verifiable degrees of freedom in monitored compu- tation. Attack vectors include steganography, un- reported modification of inference software, and covert computation via unreported batch elements. Empirically, we analyze how modern inference engines (vLLM, HF transformers) produce deter- ministic but non-invariant outputs, without need- ing to set performance-compromising determin- ism flags, if the right information is available for re-computation and no atomic functions are called in the backend. We demonstrate that such bitwise- precise re-computation does not require access to identical hardware, via a software-only emula- tion of LLM inference across multiple NVIDIA GPU variants. Thus, accumulated rounding errors can be an auditable signature of the software and hardware setup used for inference, instead of a constraint on verifiability.