Making Large Language Models Better Reasoners with Orchestrated Streaming Experiences

2025-04-01

AI Summary

Zero-shot chain-of-thought (CoT) reasoning suffers from poor performance, while few-shot approaches rely on manually crafted examples. To address this, we propose RoSE—a retrieval-augmented self-enhancement framework that improves zero-shot CoT reasoning without human annotation or model fine-tuning. RoSE dynamically constructs and maintains a streaming experience pool of historical question-answer pairs (including CoT traces), automatically retrieving and orchestrating relevant instances to augment inference. Its core innovation is a problem-aware bucketed diversity sampling mechanism that jointly optimizes for semantic similarity, prediction uncertainty, and reasoning complexity to generate high-quality prompts. RoSE enables end-to-end self-improvement via LLM-driven experience storage, multi-dimensional similarity computation, uniform bucketed sampling, and explicit complexity modeling. Extensive experiments demonstrate consistent and significant gains in zero-shot reasoning accuracy across diverse benchmark tasks, multiple large language models, and CoT variants—achieving strong generalization with minimal deployment overhead.

Technology Category

• Search and Optimization: Learning to Search
• Reasoning under Uncertainty: Sequential Decision Making
• Knowledge Representation and Reasoning: Common-Sense Reasoning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Large language models (LLMs) can perform complex reasoning by generating intermediate thoughts under zero-shot or few-shot settings. However, zero-shot prompting always encounters low performance, and the superior performance of few-shot prompting hinges on the manual-crafted demonstrations. In this paper, we present RoSE (Reasoning with Orchestrated Streaming Experiences), a general framework for solving reasoning tasks that can self-improve without complex external efforts. To enable RoSE, we describe an architecture that extends an LLM to store all answered questions and their thoughts in a streaming experience pool then orchestrates helpful questions from the pool to assist in answering new questions. To set up a question-aware orchestration mechanism, RoSE first calculates the similarity of each question in the pool with a new test question. Since the solution to each answered question is not always correct, RoSE will sort the questions according to their similarity with the new question, and then uniformly divide them into multiple buckets. It finally extracts one question from each bucket to make these extracted questions more diverse. To make these extracted questions help RoSE answer new questions as much as possible, we introduce two other attributes of uncertainty and complexity for each question. RoSE will preferentially select the questions with low uncertainty and high complexity from each bucket. We evaluate the versatility of RoSE in various reasoning tasks, LLMs, and CoT methods.

Problem

• Improving LLM reasoning without manual demonstrations
• Self-improving framework for diverse reasoning tasks
• Enhancing question selection via similarity and attributes

Innovation

• Streaming experience pool stores past Q&A
• Question-aware orchestration enhances reasoning
• Diverse questions selected by uncertainty complexity