An Empirical Study of Safetensors' Usage Trends and Developers' Perceptions

2025-01-04

AI Summary

This study investigates developers’ adoption practices and attitudes toward Hugging Face’s secure serialization format, safetensors, to assess its real-world impact on ML model serialization security. We employ a mixed-methods approach: platform metadata mining, pull request (PR) log analysis, in-depth interviews, and a developer survey. Our work is the first to systematically characterize adoption dynamics in practice—revealing that automated conversion drives widespread deployment, yet manual, intentional adoption remains low, exposing a significant tension between awareness and practical barriers. Results show sustained growth in safetensors adoption; over 50% of automated PRs are ignored or reverted; while technical challenges (e.g., compatibility) persist, developers express broadly positive willingness to learn and adapt. The study provides empirical grounding and actionable design insights for deploying secure ML tooling in practice.

Technology Category

• Machine Learning: Hardware-aware ML
• Natural Language Processing: Safety and Robustness
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Security and Privacy: Data transparency and provenance
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Developers are sharing pre-trained Machine Learning (ML) models through a variety of model sharing platforms, such as Hugging Face, in an effort to make ML development more collaborative. To share the models, they must first be serialized. While there are many methods of serialization in Python, most of them are unsafe. To tame this insecurity, Hugging Face released safetensors as a way to mitigate the threats posed by unsafe serialization formats. In this context, this paper investigates developer's shifts towards using safetensors on Hugging Face in an effort to understand security practices in the ML development community, as well as how developers react to new methods of serialization. Our results find that more developers are adopting safetensors, and many safetensor adoptions were made by automated conversions of existing models by Hugging Face's conversion tool. We also found, however, that a majority of developers ignore the conversion tool's pull requests, and that while many developers are facing issues with using safetensors, they are eager to learn about and adapt the format.

Problem

• Machine Learning Model Serialization
• Safety in AI
• Developer Attitudes

Innovation

• safetensors
• machine learning models
• developer perceptions