An Invariant Latent Space Perspective on Language Model Inversion

2025-11-24

AI Summary

Language Model Inversion (LMI) recovers sensitive prompts from model outputs, posing severe threats to user privacy and system security. To address this, we propose a novel LMI framework grounded in latent-space invariance. First, we introduce the Invariant Latent-Space Hypothesis (ILSH), identifying source and cyclic invariance as fundamental principles underlying prompt inversion. Second, we design a lightweight inverse encoder trained in two stages—without requiring large-scale inversion corpora—and optimized via training-free neighborhood search. Third, at the representation level, we fuse multi-output information through contrastive alignment, supervised reinforcement learning, sparse representation concatenation, and pseudo-representation denoising; the target LLM itself serves as an invariant decoder for efficient inverse mapping. Evaluated on nine benchmarks, our method achieves a +4.77 BLEU average gain while drastically reducing reliance on inversion data. Empirical analysis further reveals the limited efficacy of existing defenses, underscoring both the necessity and advancement of our approach.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Language model inversion (LMI), i.e., recovering hidden prompts from outputs, emerges as a concrete threat to user privacy and system security. We recast LMI as reusing the LLM's own latent space and propose the Invariant Latent Space Hypothesis (ILSH): (1) diverse outputs from the same source prompt should preserve consistent semantics (source invariance), and (2) input<->output cyclic mappings should be self-consistent within a shared latent space (cyclic invariance). Accordingly, we present Inv^2A, which treats the LLM as an invariant decoder and learns only a lightweight inverse encoder that maps outputs to a denoised pseudo-representation. When multiple outputs are available, they are sparsely concatenated at the representation layer to increase information density. Training proceeds in two stages: contrastive alignment (source invariance) and supervised reinforcement (cyclic invariance). An optional training-free neighborhood search can refine local performance. Across 9 datasets covering user and system prompt scenarios, Inv^2A outperforms baselines by an average of 4.77% BLEU score while reducing dependence on large inverse corpora. Our analysis further shows that prevalent defenses provide limited protection, underscoring the need for stronger strategies. The source code and data involved in this paper can be found in https://github.com/yyy01/Invariant_Attacker.

Problem

• Recovering hidden prompts from language model outputs threatens privacy and security
• Proposing invariant latent space hypothesis to address language model inversion
• Developing lightweight inverse encoder that reduces dependence on large corpora

Innovation

• Learns lightweight inverse encoder for denoised representations
• Uses sparse concatenation to increase information density
• Employs contrastive alignment and supervised reinforcement training