Understanding the Prompt Sensitivity

2026-04-20

AI Summary

This work investigates the sensitivity of large language models to semantically equivalent yet lexically distinct prompts, which undermines output stability. Treating the model as a multivariate function, the authors derive a theoretical upper bound on the difference in log-probabilities of the next token between semantics-preserving prompts by combining first-order Taylor expansion, gradient analysis, and the Cauchy–Schwarz inequality. Their analysis reveals that models do not internally cluster similar semantic inputs but instead represent them in a dispersed manner, preventing convergence in output behavior. Furthermore, prompt templates exert a stronger influence on logits than the underlying question content itself. The derived upper bound exhibits high correlation with PromptSensiScore, offering the first theoretical framework to explain prompt sensitivity in large language models.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Prompt sensitivity, which refers to how strongly the output of a large language model (LLM) depends on the exact wording of its input prompt, raises concerns among users about the LLM's stability and reliability. In this work, we consider LLMs as multivariate functions and perform a first-order Taylor expansion, thereby analyzing the relationship between meaning-preserving prompts, their gradients, and the log probabilities of the model's next token. We derive an upper bound on the difference between log probabilities using the Cauchy-Schwarz inequality. We show that LLMs do not internally cluster similar inputs like smaller neural networks do, but instead disperse them. This dispersing behavior leads to an excessively high upper bound on the difference of log probabilities between two meaning-preserving prompts, making it difficult to effectively reduce to 0. In our analysis, we also show which types of meaning-preserving prompt variants are more likely to introduce prompt sensitivity risks in LLMs. In addition, we demonstrate that the upper bound is strongly correlated with an existing prompt sensitivity metric, PromptSensiScore. Moreover, by analyzing the logit variance, we find that prompt templates typically exert a greater influence on logits than the questions themselves. Overall, our results provide a general interpretation for why current LLMs can be highly sensitive to prompts with the same meaning, offering crucial evidence for understanding the prompt sensitivity of LLMs. Code for experiments is available at https://github.com/ku-nlp/Understanding_the_Prompt_Sensitivity.

Problem

• prompt sensitivity
• large language models
• meaning-preserving prompts
• output stability
• log probability variance

Innovation

• prompt sensitivity
• Taylor expansion
• log probability bound
• meaning-preserving prompts
• logit variance