Understanding Before Reasoning: Enhancing Chain-of-Thought with Iterative Summarization Pre-Prompting

2025-01-08

AI Summary

Large language models (LLMs) often fail in chain-of-thought (CoT) reasoning when critical information is implicit or missing. To address this, we propose Iterative Summarization Pre-prompting (ISP²), a method grounded in the “understanding-first” paradigm. Prior to CoT reasoning, ISP² iteratively extracts, evaluates, and fuses entity-level key information via entity–description pair extraction, reliability scoring, and adaptive merging. The distilled information is then injected as a pre-prompt to guide subsequent reasoning. ISP² is plug-and-play and framework-agnostic, seamlessly integrating with diverse CoT-based inference pipelines. Evaluated across multiple complex reasoning benchmarks—including symbolic, mathematical, and commonsense reasoning—ISP² achieves an average accuracy improvement of 7.1% over state-of-the-art prompting methods. Its effectiveness demonstrates that explicit, iterative distillation of latent critical knowledge significantly enhances LLM reasoning robustness. The implementation is publicly available.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Chain-of-Thought (CoT) Prompting is a dominant paradigm in Large Language Models (LLMs) to enhance complex reasoning. It guides LLMs to present multi-step reasoning, rather than generating the final answer directly. However, CoT encounters difficulties when key information required for reasoning is implicit or missing. This occurs because CoT emphasizes the sequence of reasoning steps while overlooking the early extraction of essential information. We propose a pre-prompting method called Iterative Summarization Pre-Prompting (ISP^2) to refine LLM reasoning when key information is not explicitly provided. First, entities and their corresponding descriptions are extracted to form potential key information pairs. Next, we use a reliability rating to assess these pairs, then merge the two lowest-ranked pairs into a new entity description. This process is repeated until a unique key information pair is obtained. Finally, that pair, along with the original question, is fed into LLMs to produce the answer. Extensive experiments demonstrate a 7.1% improvement compared to existing methods. Unlike traditional prompting, ISP^2 adopts an inductive approach with pre-prompting, offering flexible integration into diverse reasoning frameworks. The code is available at https://github.com/zdhgreat/ISP-2.

Problem

• Large Language Models
• Chain of Thought
• Complex Problem Solving

Innovation

• Iterative Summary Pre-prompting
• Large Language Models
• Enhanced Accuracy