Zero Memory Overhead Approach for Protecting Vision Transformer Parameters

2025-07-04

AI Summary

Vision Transformers (ViTs) exhibit significant accuracy degradation under memory bit-flip faults—critical in safety-critical applications such as autonomous driving. Method: Leveraging the inherent sparsity and high proportion of near-zero values in ViT parameters, we propose a zero-memory-overhead fault-tolerance mechanism. Specifically, we dynamically repurpose the least-significant bit (LSB) of each parameter as a parity bit for fine-grained error detection; upon fault detection, anomalous parameters are zeroed to prevent error propagation. Contribution/Results: This work presents the first hardware-level, parameter-wise fault protection for ViTs without any additional storage overhead. It improves robustness against single-bit flips by up to three orders of magnitude compared to baseline models, substantially outperforming existing redundancy-based or checkpointing approaches. The method is lightweight, fully compatible with standard ViT inference, and suitable for high-reliability embedded AI systems.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Hardware-aware ML
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Vision Transformers (ViTs) have demonstrated superior performance over Convolutional Neural Networks (CNNs) in various vision-related tasks such as classification, object detection, and segmentation due to their use of self-attention mechanisms. As ViTs become more popular in safety-critical applications like autonomous driving, ensuring their correct functionality becomes essential, especially in the presence of bit-flip faults in their parameters stored in memory. In this paper, a fault tolerance technique is introduced to protect ViT parameters against bit-flip faults with zero memory overhead. Since the least significant bits of parameters are not critical for model accuracy, replacing the LSB with a parity bit provides an error detection mechanism without imposing any overhead on the model. When faults are detected, affected parameters are masked by zeroing out, as most parameters in ViT models are near zero, effectively preventing accuracy degradation. This approach enhances reliability across ViT models, improving the robustness of parameters to bit-flips by up to three orders of magnitude, making it an effective zero-overhead solution for fault tolerance in critical applications.

Problem

• Protecting ViT parameters from bit-flip faults
• Zero memory overhead for fault tolerance
• Enhancing ViT reliability in safety-critical applications

Innovation

• Zero memory overhead fault tolerance technique
• Replace LSB with parity bit for error detection
• Mask faulty parameters by zeroing out