Representation Consistency for Accurate and Coherent LLM Answer Aggregation

2025-06-17

arXiv.org

AI Summary

This work addresses the limitation in test-time scaling where multi-candidate answer aggregation relies solely on output frequency, ignoring internal model consistency. We propose Representation Consistency (RC), a method that leverages cached LLM internal activations—without modifying prompts or sampling strategies—to assess candidate answer agreement in the representation space via lightweight similarity computation. RC is the first to use internal representational consistency—not output frequency—as the aggregation criterion. It supports both dense and sparse activations and reveals that sparse activation consistency strongly correlates with coherent reasoning. RC integrates pretrained sparse autoencoders with a query-free activation reuse mechanism. Evaluated across four open-source LLMs and four reasoning benchmarks, RC achieves up to a 4% absolute accuracy gain over state-of-the-art test-time scaling baselines, demonstrating significant and consistent improvements.

Technology Category

• Knowledge Representation and Reasoning: Computational Complexity of Reasoning
• Reasoning under Uncertainty: Uncertainty Representations
• Machine Learning: Active Learning

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Test-time scaling improves large language models'(LLMs) performance by allocating more compute budget during inference. To achieve this, existing methods often require intricate modifications to prompting and sampling strategies. In this work, we introduce representation consistency (RC), a test-time scaling method for aggregating answers drawn from multiple candidate responses of an LLM regardless of how they were generated, including variations in prompt phrasing and sampling strategy. RC enhances answer aggregation by not only considering the number of occurrences of each answer in the candidate response set, but also the consistency of the model's internal activations while generating the set of responses leading to each answer. These activations can be either dense (raw model activations) or sparse (encoded via pretrained sparse autoencoders). Our rationale is that if the model's representations of multiple responses converging on the same answer are highly variable, this answer is more likely to be the result of incoherent reasoning and should be down-weighted during aggregation. Importantly, our method only uses cached activations and lightweight similarity computations and requires no additional model queries. Through experiments with four open-source LLMs and four reasoning datasets, we validate the effectiveness of RC for improving task performance during inference, with consistent accuracy improvements (up to 4%) over strong test-time scaling baselines. We also show that consistency in the sparse activation signals aligns well with the common notion of coherent reasoning.

Problem

• Improves LLM answer aggregation using representation consistency
• Enhances accuracy by weighting answers with consistent activations
• Requires no additional model queries during inference scaling

Innovation

• Aggregates answers using internal activation consistency
• Uses dense or sparse model representations for weighting
• Requires no additional queries, only cached activations