🤖 AI Summary
This study investigates whether subjective experience reports generated by large language models under self-referential prompts exhibit response consistency, and compares their stability against those elicited by intractable philosophical questions and verifiable factual questions. Through 30 independent sampling trials, output instability is quantified using sentence embeddings and mean pairwise cosine similarity. The findings reveal, for the first time quantitatively, that self-referential prompts yield significantly unstable subjective reports (similarity: 0.343 ± 0.047), markedly higher than both philosophical (0.192 ± 0.008) and verifiable questions (0.105 ± 0.058). This suggests that the observed uncertainty stems from a distinct mechanism induced by self-reference rather than general question openness. Experiments were conducted using the Gemini API (temperature = 0.7) with systematic prompt engineering.
📝 Abstract
Self-referential prompting has been shown to reliably induce large language models to produce first-person reports resembling subjective experience, but no prior work measures how consistent these reports are across repeated, independent trials, or how that consistency compares to the model's behavior on other kinds of open-ended questions. We measure response instability, defined as one minus the mean pairwise cosine similarity of sentence embeddings computed over a compressed core claim extracted from each response, for three groups of questions: self-referential prompts eliciting a subjective-experience report, unresolvable philosophical questions unrelated to self-reference, and questions with a verifiable correct answer. Using 30 independent responses per question (360 responses total, Gemini API, temperature 0.7) across four questions per group, we find that self-referential questions show the highest instability (0.343 +/- 0.047), unresolvable philosophy questions show intermediate and tightly clustered instability (0.192 +/- 0.008), and verifiable questions show the lowest instability (0.105 +/- 0.058). This provides a quantitative baseline for the induced subjective-experience report, showing that it occupies a distinct, less stable position in the model's output distribution than ordinary open-ended philosophical uncertainty.