What do your logits know? (The answer may surprise you!)

2026-04-10

AI Summary

This work investigates the privacy risks posed by visual-language models, demonstrating that their outputs—such as top-k logits—can inadvertently leak task-irrelevant sensitive information present in input images. The study presents the first systematic evaluation of information retention across different representational layers, including residual streams, tuned lens projections, and logits. By integrating tuned lens analysis, residual stream inspection, and quantitative information-theoretic measures, the authors conduct an empirical assessment revealing that top logits alone can recover nearly as much sensitive information as the full residual stream. These findings uncover a significant risk of high-dimensional information leakage through lightweight model outputs, offering critical insights for the secure deployment of multimodal foundation models.

Technology Category

• Computer Vision: Large Vision Models
• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Security and Privacy: Data transparency and provenance
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Large language models for search

Abstract

Recent work has shown that probing model internals can reveal a wealth of information not apparent from the model generations. This poses the risk of unintentional or malicious information leakage, where model users are able to learn information that the model owner assumed was inaccessible. Using vision-language models as a testbed, we present the first systematic comparison of information retained at different"representational levels''as it is compressed from the rich information encoded in the residual stream through two natural bottlenecks: low-dimensional projections of the residual stream obtained using tuned lens, and the final top-k logits most likely to impact model's answer. We show that even easily accessible bottlenecks defined by the model's top logit values can leak task-irrelevant information present in an image-based query, in some cases revealing as much information as direct projections of the full residual stream.

Problem

• information leakage
• vision-language models
• logits
• residual stream
• privacy risk

Innovation

• information leakage
• vision-language models
• residual stream
• logits
• probing