Breaking the Pre-Sampling Barrier: Activation-Informed Difficulty-Aware Self-Consistency

2026-02-10

AI Summary

This work addresses the high computational cost of self-consistency reasoning, which stems from its reliance on extensive sampling, and the additional overhead incurred by existing difficulty-adaptive methods that require pre-sampling to estimate problem difficulty. To overcome these limitations, we propose a lightweight difficulty estimation probe that dynamically adjusts the number of self-consistency samples by analyzing neuron activation patterns within the feedforward networks of large language models. Our approach is the first to leverage internal activations for pre-sampling-free, difficulty-aware adaptation, enabling plug-and-play deployment on new datasets without extra model calls or pre-sampling steps. Evaluated across five benchmark datasets, the method significantly reduces computational cost while maintaining competitive reasoning accuracy, outperforming current adaptive sampling strategies.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Machine Learning: Active Learning
• Knowledge Representation and Reasoning: Computational Complexity of Reasoning

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Self-Consistency (SC) is an effective decoding strategy that improves the reasoning performance of Large Language Models (LLMs) by generating multiple chain-of-thought reasoning paths and selecting the final answer via majority voting. However, it suffers from substantial inference costs because it requires a large number of samples. To mitigate this issue, Difficulty-Adaptive Self-Consistency (DSC) was proposed to reduce unnecessary token usage for easy problems by adjusting the number of samples according to problem difficulty. However, DSC requires additional model calls and pre-sampling to estimate difficulty, and this process is repeated when applying to each dataset, leading to significant computational overhead. In this work, we propose Activation-Informed Difficulty-Aware Self-Consistency (ACTSC) to address these limitations. ACTSC leverages internal difficulty signals reflected in the feed-forward network neuron activations to construct a lightweight difficulty estimation probe, without any additional token generation or model calls. The probe dynamically adjusts the number of samples for SC and can be applied to new datasets without requiring pre-sampling for difficulty estimation. To validate its effectiveness, we conduct experiments on five benchmarks. Experimental results show that ACTSC effectively reduces inference costs while maintaining accuracy relative to existing methods.

Problem

• Self-Consistency
• inference cost
• difficulty estimation
• pre-sampling
• Large Language Models

Innovation

• Self-Consistency
• difficulty estimation
• neuron activation
• inference efficiency
• large language models