Amortized Latent Steering: Low-Cost Alternative to Test-Time Optimization

2025-09-10

arXiv.org

AI Summary

Test-time optimization (TTO) incurs prohibitively high inference overhead—10–100× that of standard decoding—due to iterative optimization and multi-step backpropagation, hindering practical deployment. To address this, we propose **Amortized Latent Steering (ALS)**, a gradient-free, constant-overhead alternative that eliminates online optimization. ALS precomputes, offline, the mean difference vector between latent states of successful and failed examples, enabling efficient, zero-gradient latent-space calibration at inference time. By integrating directional latent-space steering with a mean-field approximation, ALS achieves 2–5× speedup over iterative TTO on GSM8K and MATH-500 while improving the efficiency–accuracy trade-off by 101%. It matches or surpasses greedy chain-of-thought and Self-Consistency baselines in accuracy, marking the first practical, low-latency, high-accuracy deployment of TTO without backpropagation.

Technology Category

• Search and Optimization: Learning to Search
• Planning, Routing, and Scheduling: Planning with Language Models
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Test-time optimization remains impractical at scale due to prohibitive inference costs--techniques like iterative refinement and multi-step verification can require $10-100 imes$ more compute per query than standard decoding. Latent space test-time optimization methods like LatentSeek offer a more direct approach by steering hidden representations, but still demand expensive per-query optimization loops with multiple backward passes. We propose Amortized Latent Steering (ALS), which collapses this iterative optimization into a single offline-computed vector applied at constant cost during inference. ALS computes the mean difference between hidden states from successful versus unsuccessful generations, then uses this direction to calibrate the model's hidden representations: when decoding drifts away from the success manifold, ALS nudges activations back toward it. Across GSM8K and MATH-500 benchmarks, ALS achieves $2-5 imes$ speedup over iterative methods while matching or surpassing greedy Chain-of-Thought (CoT) and Self-Consistency baselines, yielding up to 101% improvement in efficiency--accuracy trade-off. These results show that much of latent optimization's benefit can be captured offline, making sophisticated reasoning techniques viable for production deployment. Code is available at https://github.com/negbuna/ALS.

Problem

• Reducing high inference costs of test-time optimization methods
• Eliminating expensive per-query optimization loops in latent steering
• Making sophisticated reasoning techniques viable for production deployment

Innovation

• Amortized Latent Steering replaces iterative optimization loops
• Uses single offline-computed vector for constant-cost inference
• Steers hidden representations toward successful generation manifold