P-CoT: A Pedagogically-motivated Participatory Chain-of-Thought Prompting for Phonological Reasoning in LLMs

2025-07-22

AI Summary

This study investigates the capacity of purely text-based large language models (LLMs) to perform phonological reasoning tasks—including rhyme generation, grapheme-to-phoneme conversion, and syllable counting—despite their lack of explicit phonetic representations. To address this limitation, we propose Participatory Chain-of-Thought (P-CoT) prompting, a pedagogically grounded method integrating scaffolding instruction and discovery learning. P-CoT guides models through structured, educationally informed reasoning steps to elicit latent phonological processing capabilities. We evaluate 12 LLMs under few-shot settings on the PhonologyBench benchmark. Results show that P-CoT yields an average performance gain of 52% across tasks, with several surpassing human baseline accuracy. This work constitutes the first systematic demonstration that text-only LLMs possess latent, elicitable phonological reasoning abilities. It establishes a novel paradigm for modeling phonological cognition in language models and advances applications in educational AI.

Technology Category

• Natural Language Processing: (Large) Language Models
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

This study explores the potential of phonological reasoning within text-based large language models (LLMs). Utilizing the PhonologyBench benchmark, we assess tasks like rhyme word generation, g2p conversion, and syllable counting. Our evaluations across 12 LLMs reveal that while few-shot learning offers inconsistent gains, the introduction of a novel Pedagogically-motivated Participatory Chain-of-Thought (P-CoT) prompt, which is anchored in educational theories like scaffolding and discovery learning, consistently enhances performance. This method leverages structured guidance to activate latent phonological abilities, achieving up to 52% improvement and even surpassing human baselines in certain tasks. Future work could aim to optimize P-CoT prompts for specific models or explore their application across different linguistic domains.

Problem

• Enhancing phonological reasoning in LLMs using P-CoT prompts
• Evaluating LLMs on rhyme, g2p, and syllable tasks via PhonologyBench
• Improving performance via educational theories like scaffolding

Innovation

• Pedagogically-motivated Chain-of-Thought prompting
• Scaffolding and discovery learning integration
• Structured guidance for phonological reasoning