GIF: Locally Sound Geometric Information Flow Control for LLMs

2026-06-22

AI Summary

Large language models (LLMs) deployed in agent systems face critical security risks such as prompt injection and privacy leakage. Existing information flow control methods, lacking a semantic foundation, often suffer from taint explosion. To address this, this work proposes the Geometric Information Flow (GIF) framework, which introduces—for the first time—a semantic information flow metric grounded in the Jacobian matrix of LLMs and the local geometric structure of their outputs, ensuring local geometric fidelity. Under a local regularity assumption, GIF’s strict upper bound on true Shannon mutual information is formally verified in Lean 4. Integrating automatic differentiation, low-rank approximation, and a lightweight declassifier, GIF achieves near-perfect recall across diverse attack benchmarks, matches the F1 scores of strong baselines such as GPT-5.5, reduces inference cost by 81×, and demonstrates transferable detection capability from smaller to larger model families.

Abstract

Large language models increasingly mediate interactions between sensitive data, untrusted inputs, and privileged actions in agentic systems, creating security and privacy risks. These range from prompt injections that manipulate downstream tool use to leakage of confidential information through model outputs. Recent Information Flow Control (IFC)-based defenses show promise but lack a principled semantic foundation for reasoning about information flow through the model itself. Since any input token may influence any output token in an autoregressive LLM, existing approaches suffer from severe taint explosion. We present Geometric Information Flow (GIF), a semantic framework for tracking information flow from input tokens to outputs. GIF uses the LLM Jacobian and local output geometry to upper-bound the Shannon mutual information between perturbed input spans and model outputs, yielding a scalable measure computable on large models via automatic differentiation and low-rank approximation. Unlike attention-based or correlational attribution heuristics, GIF satisfies local geometric soundness, and we provide a fully mechanized Lean 4 proof that it upper-bounds the true information flow induced by a given prompt under local regularity assumptions. We evaluate GIF on integrity and confidentiality tasks across multiple prompt-injection and privacy-leakage benchmarks. GIF achieves near-perfect recall even without a downstream declassifier, outperforming attention-based baselines. Combined with lightweight LLM-based declassifiers, it matches or exceeds the F1 of direct LLM-as-judge baselines such as GPT-5.5 xhigh reasoning while using up to 81x lower token cost. GIF flows detected with small surrogate models transfer to larger state-of-the-art models and other model families, even when the surrogate is up to 200x smaller, suggesting black-box deployment without gradient access.

Problem

• Information Flow Control
• Large Language Models
• Security
• Privacy
• Prompt Injection

Innovation

• Geometric Information Flow
• Information Flow Control
• Jacobian-based Attribution
• Local Geometric Soundness
• Low-rank Approximation