Structured Thoughts For Improved Reasoning And Context Pruning

2026-07-11

AI Summary

This work addresses the limitations of long reasoning chains generated by large language models, which are often constrained by context length and memory inefficiency. The authors propose a structured reasoning framework that explicitly alternates between <try> blocks for exploratory drafts and <outcome> blocks for refined conclusions. Through supervised fine-tuning, the model is guided to adhere to this structured format, enabling dynamic pruning of redundant contextual information while preserving essential conclusions. Experimental results demonstrate that this approach improves reasoning performance by up to 8.08% compared to standard supervised fine-tuning. Furthermore, the proposed pruning strategy reduces context memory usage by an average of 85%, with only an 8.67% relative drop in performance, thereby achieving a favorable trade-off between efficiency and accuracy.

Abstract

Large language models (LLMs) excel at generating long chains of thought, but long reasoning traces are often verbose and memory-inefficient. In this work, we introduce Structured Thoughts, a framework that organizes reasoning into alternating <try> and <outcome> blocks: <try> captures exploratory scratch work, while <outcome> contains the distilled conclusion of that step. We construct a dataset of structured thoughts by segmenting reasoning traces into <try> blocks and prompting an LLM to summarize each step into its corresponding <outcome>. Fine-tuning pretrained foundation models on this reformatted data produces models that adopt the structured reasoning style, leading to performance gains of up to 8.08\% on reasoning benchmarks compared to standard SFT. The explicit structure also enables context pruning: after each <try>/<outcome> pair, the <try> can be pruned, allowing the model to retain conclusions without keeping the full scratch work in the context. A proof-of-concept pruning implementation achieves an average of 85\% memory / context savings with an 8.67\% performance drop across mathematical tasks.

Problem

• reasoning efficiency
• context pruning
• memory inefficiency
• verbose reasoning
• structured reasoning

Innovation

• Structured Thoughts
• reasoning efficiency
• context pruning
• chain-of-thought
• memory optimization