Toward Open Weight Models Without Risks: Separating Public and Private Capabilities in LLMs

2026-06-19

AI Summary

This work addresses the challenge of safely controlling sensitive capabilities in open-weight large language models without compromising openness. The authors propose Tiered Language Models (TLM), which, for the first time, enable separation of public and private capabilities within a single set of model weights. By default, the model operates in a public mode indistinguishable from standard LLMs, while a compact cryptographic key activates a private computational pathway to unlock enhanced functionalities. This mechanism embeds authorization directly into the weight structure rather than the input space, inherently resisting capability extraction via fine-tuning and partial key leakage, and supporting multi-tier capability expansion. Experiments on 180M and 650M parameter models demonstrate that private modes can acquire new languages, instruction-following skills, and proprietary knowledge—capabilities entirely absent in public modes—while maintaining strong robustness against adversarial attacks.

Abstract

Open-weight Large Language Models (LLMs) enable scientific progress and broad deployment. However, they make it difficult to control access to sensitive capabilities. Current practice either suppresses dangerous capabilities before release or mediates access through closed services that use specialized model variants, input/output monitors, and API permissions. The former is susceptible to jailbreaks while sacrificing capability for all users to mitigate the risks posed by a few, and the latter is fundamentally incompatible with open-weight release. In this paper, we propose Tiered Language Models (TLMs), where a single set of released weights supports multiple capability levels. In its default public configuration, a TLM behaves as a conventional LLM. A compact secret key specifies a permutation over a small parameter subset, inducing an alternative computation graph over the same weights that exposes additional capabilities. We develop a training protocol that jointly pretrains both configurations from scratch, then fine-tunes the keyed configuration on private data with regularization to preserve the public model's behavior. We pretrain 180M- and 650M-parameter TLMs and demonstrate that the keyed configuration can acquire a new language, gain instruction-following ability, and memorize private factual knowledge, whereas the public configuration exhibits none of these capabilities. Moreover, we show that our approach extends naturally to multiple hierarchical tiers. Because authorization operates on the model's weight structure rather than in the input space, the mechanism resists fine-tuning-based extraction and partial key compromise. In general, TLMs take a step toward reconciling open-weight release with selective capability control.

Problem

• open-weight LLMs
• capability control
• private capabilities
• model security
• selective access

Innovation

• Tiered Language Models
• open-weight LLMs
• capability control
• parameter permutation
• secret key activation