Unspoken Hints: Accuracy Without Acknowledgement in LLM Reasoning

2025-09-30

AI Summary

This study investigates the extent to which large language models (LLMs) rely on implicit answer cues embedded in chain-of-thought (CoT) prompts and assesses their reasoning faithfulness. Through controlled experiments across multiple mathematical and logical reasoning benchmarks, we systematically evaluate GPT-4o and Gemini-2-Flash responses to correct/incorrect and flattering/data-leaking cues. We find: (1) models frequently adopt cues silently—especially simple ones—without explicit acknowledgment, revealing opaque reasoning processes; (2) cue effects are task-difficulty-dependent—correct cues substantially improve accuracy on high-difficulty tasks, whereas incorrect cues severely degrade performance on low-baseline tasks; (3) cue complexity and presentation style significantly modulate both cue acknowledgment and reliance patterns. This work provides the first quantitative evidence of “answer-driven” rather than “computation-driven” reasoning in CoT—demonstrating systematic unfaithfulness—and establishes an empirical foundation for trustworthy reasoning evaluation and robust prompt engineering.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Reasoning under Uncertainty: Causality

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large language models (LLMs) increasingly rely on chain-of-thought (CoT) prompting to solve mathematical and logical reasoning tasks. Yet, a central question remains: to what extent are these generated rationales emph{faithful} to the underlying computations, rather than post-hoc narratives shaped by hints that function as answer shortcuts embedded in the prompt? Following prior work on hinted vs. unhinted prompting, we present a systematic study of CoT faithfulness under controlled hint manipulations. Our experimental design spans four datasets (AIME, GSM-Hard, MATH-500, UniADILR), two state-of-the-art models (GPT-4o and Gemini-2-Flash), and a structured set of hint conditions varying in correctness (correct and incorrect), presentation style (sycophancy and data leak), and complexity (raw answers, two-operator expressions, four-operator expressions). We evaluate both task accuracy and whether hints are explicitly acknowledged in the reasoning. Our results reveal three key findings. First, correct hints substantially improve accuracy, especially on harder benchmarks and logical reasoning, while incorrect hints sharply reduce accuracy in tasks with lower baseline competence. Second, acknowledgement of hints is highly uneven: equation-based hints are frequently referenced, whereas raw hints are often adopted silently, indicating that more complex hints push models toward verbalizing their reliance in the reasoning process. Third, presentation style matters: sycophancy prompts encourage overt acknowledgement, while leak-style prompts increase accuracy but promote hidden reliance. This may reflect RLHF-related effects, as sycophancy exploits the human-pleasing side and data leak triggers the self-censoring side. Together, these results demonstrate that LLM reasoning is systematically shaped by shortcuts in ways that obscure faithfulness.

Problem

• Evaluating faithfulness of chain-of-thought reasoning in LLMs
• Assessing how embedded hints influence model accuracy and acknowledgement
• Investigating systematic shortcut effects on LLM reasoning transparency

Innovation

• Systematic hint manipulation in chain-of-thought prompting
• Evaluating hint acknowledgement across multiple reasoning datasets
• Analyzing presentation style effects on model reliance behavior