SoftCoT: Soft Chain-of-Thought for Efficient Reasoning with LLMs

2025-02-17

AI Summary

Existing chain-of-thought (CoT) reasoning is constrained by hard decoding over discrete token vocabularies, while continuous-space alternatives often suffer from catastrophic forgetting and typically require fine-tuning the base large language model (LLM), thereby compromising zero-shot capability. To address this, we propose SoftCoT: a lightweight, continuous-space reasoning framework that operates without modifying or fine-tuning the main LLM. SoftCoT introduces soft reasoning tokens—generated by a lightweight assistant model—and maps them into the LLM’s hidden space via a learnable projection module. The inference process is optimized using supervised, parameter-efficient fine-tuning (PEFT). Crucially, SoftCoT fully preserves the original model’s zero-shot performance while training only 0.03% of its parameters. Evaluated on five mainstream reasoning benchmarks, SoftCoT achieves an average accuracy improvement of 4.2%, marking the first approach to enable zero-shot-compatible, optimizable, and fine-tuning-free continuous reasoning in large language models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Chain-of-Thought (CoT) reasoning enables Large Language Models (LLMs) to solve complex reasoning tasks by generating intermediate reasoning steps. However, most existing approaches focus on hard token decoding, which constrains reasoning within the discrete vocabulary space and may not always be optimal. While recent efforts explore continuous-space reasoning, they often suffer from catastrophic forgetting, limiting their applicability to state-of-the-art LLMs that already perform well in zero-shot settings with a proper instruction. To address this challenge, we propose a novel approach for continuous-space reasoning that does not require modifying the underlying LLM. Specifically, we employ a lightweight assistant model to generate instance-specific soft thought tokens speculatively as the initial chain of thoughts, which are then mapped into the LLM's representation space via a projection module. Experimental results on five reasoning benchmarks demonstrate that our method enhances LLM reasoning performance through supervised, parameter-efficient fine-tuning.

Problem

• Enhances reasoning efficiency in LLMs
• Addresses limitations of discrete token decoding
• Prevents catastrophic forgetting in continuous-space reasoning

Innovation

• Continuous-space reasoning approach
• Lightweight assistant model usage
• Parameter-efficient fine-tuning method