What's the Difference? Supporting Users in Identifying the Effects of Prompt and Model Changes Through Token Patterns

2025-04-22

AI Summary

This study addresses the limited output interpretability in large language model (LLM) prompt engineering by proposing an automated, semantics-oriented differential analysis method. Methodologically, it introduces a novel data-mining–based token-sequence pattern extraction technique to reliably distinguish systematic output differences—caused by prompt or model modifications—from stochastic decoding variations; it further constructs three benchmark test suites covering sensitive dimensions (e.g., gender, culture) to enable bias detection. Contributions include: (1) the first token-level framework for systematic difference detection; (2) empirical validation across multiple prompt engineering datasets demonstrating high reliability of pattern extraction; and (3) user studies confirming significant improvements in both efficiency and depth of human understanding of model behavioral differences—thereby enabling human-centered model behavior analysis and controllable prompt engineering.

Technology Category

• Natural Language Processing: Prompt Engineering / Prompting
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Prompt engineering for large language models is challenging, as even small prompt perturbations or model changes can significantly impact the generated output texts. Existing evaluation methods, either automated metrics or human evaluation, have limitations, such as providing limited insights or being labor-intensive. We propose Spotlight, a new approach that combines both automation and human analysis. Based on data mining techniques, we automatically distinguish between random (decoding) variations and systematic differences in language model outputs. This process provides token patterns that describe the systematic differences and guide the user in manually analyzing the effects of their prompt and model changes efficiently. We create three benchmarks to quantitatively test the reliability of token pattern extraction methods and demonstrate that our approach provides new insights into established prompt data. From a human-centric perspective, through demonstration studies and a user study, we show that our token pattern approach helps users understand the systematic differences of language model outputs, and we are able to discover relevant differences caused by prompt and model changes (e.g. related to gender or culture), thus supporting the prompt engineering process and human-centric model behavior research.

Problem

• Identifying effects of prompt and model changes
• Limitations of existing evaluation methods
• Supporting prompt engineering and human analysis

Innovation

• Automated token pattern extraction for systematic differences
• Combines data mining with human analysis efficiently
• Benchmarks validate reliability of pattern extraction