A Light-Weight Large Language Model File Format for Highly-Secure Model Distribution

2025-12-04

AI Summary

Securely distributing and deploying large language models (LLMs) in open environments—while ensuring weight confidentiality and execution control without relying on costly encryption or private infrastructure—remains a critical challenge. Method: This paper introduces CryptoTensors, a lightweight, Safetensors-based secure model file format. Its core innovation is the first integration of tensor-level fine-grained encryption, embedded policy-driven access control, and automated key management directly within the model file, enabling lazy loading and partial deserialization. Contribution/Results: CryptoTensors achieves transparent compatibility with mainstream inference frameworks—including Hugging Face Transformers and vLLM—without requiring modifications to existing pipelines. It delivers end-to-end secure deployment with near-zero runtime overhead, significantly enhancing both privacy guarantees and engineering practicality for LLMs in untrusted environments.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Computer Vision: Large Vision Models
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Cryptocurrency and smart contracts
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

To enhance the performance of large language models (LLMs) in various domain-specific applications, sensitive data such as healthcare, law, and finance are being used to privately customize or fine-tune these models. Such privately adapted LLMs are regarded as either personal privacy assets or corporate intellectual property. Therefore, protecting model weights and maintaining strict confidentiality during deployment and distribution have become critically important. However, existing model formats and deployment frameworks provide little to no built-in support for confidentiality, access control, or secure integration with trusted hardware. Current methods for securing model deployment either rely on computationally expensive cryptographic techniques or tightly controlled private infrastructure. Although these approaches can be effective in specific scenarios, they are difficult and costly for widespread deployment. In this paper, we introduce CryptoTensors, a secure and format-compatible file structure for confidential LLM distribution. Built as an extension to the widely adopted Safetensors format, CryptoTensors incorporates tensor-level encryption and embedded access control policies, while preserving critical features such as lazy loading and partial deserialization. It enables transparent decryption and automated key management, supporting flexible licensing and secure model execution with minimal overhead. We implement a proof-of-concept library, benchmark its performance across serialization and runtime scenarios, and validate its compatibility with existing inference frameworks, including Hugging Face Transformers and vLLM. Our results highlight CryptoTensors as a light-weight, efficient, and developer-friendly solution for safeguarding LLM weights in real-world and widespread deployments.

Problem

• Securing sensitive LLM weights during distribution and deployment
• Providing built-in confidentiality and access control for models
• Enabling lightweight security without expensive cryptographic overhead

Innovation

• Extends Safetensors with tensor-level encryption and access control
• Enables transparent decryption and automated key management
• Lightweight format compatible with existing inference frameworks