Process In-Context Learning: Enhancing Mathematical Reasoning via Dynamic Demonstration Insertion

2026-01-17

AI Summary

This work addresses the limitations of static in-context learning in multi-step mathematical reasoning, where fixed demonstrations fail to adapt to emerging points of confusion during inference, often leading to cascading errors. To overcome this, the authors propose Processual In-Context Learning (PICL), a novel framework that dynamically inserts relevant examples on demand during the reasoning process. PICL identifies confusion points through semantic analysis and entropy-based uncertainty detection, then retrieves the most pertinent demonstrations from a candidate pool for real-time contextual injection. This approach breaks free from the constraints of traditional static paradigms, significantly outperforming existing baselines across multiple mathematical reasoning benchmarks. By adaptively clarifying ambiguities as they arise, PICL effectively mitigates mid-process confusion and enhances reasoning accuracy in complex logical tasks.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Reasoning under Uncertainty: Other Foundations of Reasoning under Uncertainty
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

In-context learning (ICL) has proven highly effective across diverse large language model (LLM) tasks. However, its potential for enhancing tasks that demand step-by-step logical deduction, such as mathematical reasoning, remains underexplored. A core limitation of existing ICL approaches is their static use of demonstrations: examples are pre-selected before inference and remain fixed, failing to adapt to the dynamic confusion points that often arise during multi-step reasoning such as ambiguous calculations or logical gaps. These unresolved confusion points can lead to cascading errors that degrade final accuracy. To tackle this issue, we propose Process In-Context Learning (PICL), a dynamic demonstration integration framework designed to boost mathematical reasoning by responding to real-time inference needs. PICL operates in two stages: 1)~it identifies potential confusion points by analyzing semantics and entropy in the reasoning process and summarizes their core characteristics; 2)~upon encountering these points, it retrieves relevant demonstrations from the demonstration pool that match the confusion context and inserts them directly into the ongoing reasoning process to guide subsequent steps. Experiments show that PICL outperforms baseline methods by mitigating mid-inference confusion, highlighting the value of adaptive demonstration insertion in complex mathematical reasoning.

Problem

• In-context learning
• Mathematical reasoning
• Dynamic demonstration
• Confusion points
• Step-by-step deduction

Innovation

• Process In-Context Learning
• Dynamic Demonstration Insertion
• Mathematical Reasoning
• Confusion Point Detection
• Adaptive Inference