SPARK: Susceptibility-Guided Profiling and Steering of Latent Reasoning States in Large Language Models

2026-07-11

AI Summary

Existing approaches struggle to precisely diagnose the underlying mechanisms behind reasoning failures in large language models and lack effective, sample-specific intervention strategies. This work proposes an analysis framework based on hidden-state susceptibility, which incorporates input-length control to eliminate confounding effects from sequence length and leverages a cross-layer coordination mechanism to accurately identify critical reasoning states. The framework enables lightweight, test-time guidance that significantly enhances model performance. Evaluated on the MATH-500 benchmark, it improves the accuracy of Qwen3-4B and Qwen3-8B to 84.6% and 85.6%, respectively—substantially outperforming current baselines—and thereby demonstrates its effectiveness in both diagnosing failure modes and enabling targeted interventions.

Abstract

Reasoning failures in large language models (LLMs) are usually evaluated from final answers, but a wrong answer does not reveal why the model failed. The same incorrect output may reflect missing capability, an unstable reasoning trajectory, or a failure to activate a reasoning state that is already available in the frozen model. Existing prompting and benchmark-based evaluation methods mostly operate at the output level, while generic activation-steering methods typically apply global directions without diagnosing which examples require intervention. In this paper, we introduce SPARK, which uses hidden-state response to diagnose whether a model internally enters an effective reasoning state and to guide lightweight test-time steering. The key observation is that raw hidden-state susceptibility is strongly confounded by prompt length, especially in programmatic and algorithmic reasoning where harder serialized instances naturally become longer. SPARK therefore uses length-controlled susceptibility to separate input-scale effects from residual reasoning activation, and combines this signal with cross-layer coordination to select reasoning-active anchors and under-activated hard examples. We use FRONTIER-4.5K as a controlled programmatic reasoning suite for latent profiling and difficulty-aware analysis, and evaluate SPARK-Steering on GSM8K and MATH-500 with forward-only benchmark profiling. Our method improves Qwen3 series models consistently; on MATH-500, accuracy rises from 82.0% to 84.6% for Qwen3-4B and from 82.4% to 85.6% for Qwen3-8B. These results suggest that susceptibility can serve not only as a diagnostic signal for reasoning failures, but also as a practical guide for targeted test-time intervention.

Problem

• reasoning failures
• latent reasoning states
• susceptibility
• test-time steering
• hidden-state activation

Innovation

• susceptibility-guided steering
• latent reasoning states
• length-controlled susceptibility
• test-time intervention
• hidden-state profiling