Activation Control for Efficiently Eliciting Long Chain-of-thought Ability of Language Models

2025-05-23

AI Summary

How to elicit chain-of-thought (CoT) reasoning in large language models (LLMs) without reinforcement learning or supervised fine-tuning. Method: We propose a training-free method that modulates activations of final-layer neurons. Our key insight is that CoT capability is governed by a sparse set of high-impact neurons in the output layer. Accordingly, we introduce a three-component mechanism: (i) contrastive exemplar–driven identification of critical neurons; (ii) real-time activation modulation via analytical functions (e.g., exponential decay or step functions); and (iii) injection of “wait” tokens to extend reasoning steps. We further combine lightweight LoRA adaptation with single-layer activation amplification. Results: The method achieves zero-shot CoT performance surpassing supervised fine-tuning baselines. It uses only 12% of full LoRA parameters while significantly improving self-reflection rates and reasoning accuracy. Notably, it enables the first predictive and interpretable modeling of neuron activation trajectories during inference.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Despite the remarkable reasoning performance, eliciting the long chain-of-thought (CoT) ability in large language models (LLMs) typically requires costly reinforcement learning or supervised fine-tuning on high-quality distilled data. We investigate the internal mechanisms behind this capability and show that a small set of high-impact activations in the last few layers largely governs long-form reasoning attributes, such as output length and self-reflection. By simply amplifying these activations and inserting"wait"tokens, we can invoke the long CoT ability without any training, resulting in significantly increased self-reflection rates and accuracy. Moreover, we find that the activation dynamics follow predictable trajectories, with a sharp rise after special tokens and a subsequent exponential decay. Building on these insights, we introduce a general training-free activation control technique. It leverages a few contrastive examples to identify key activations, and employs simple analytic functions to modulate their values at inference time to elicit long CoTs. Extensive experiments confirm the effectiveness of our method in efficiently eliciting long CoT reasoning in LLMs and improving their performance. Additionally, we propose a parameter-efficient fine-tuning method that trains only a last-layer activation amplification module and a few LoRA layers, outperforming full LoRA fine-tuning on reasoning benchmarks with significantly fewer parameters. Our code and data are publicly released.

Problem

• Eliciting long chain-of-thought in LLMs without costly training
• Identifying high-impact activations controlling long-form reasoning attributes
• Developing training-free activation control for efficient reasoning enhancement

Innovation

• Amplify key activations for long CoT
• Use wait tokens to enhance reasoning
• Train-free activation control technique