Toward Latent Language Model Skills Steering and Optimization: An Empirical Study

2026-08-29

AI Summary

研究探讨了通过激活空间中的向量表示来操纵大型语言模型的程序技能，发现这些技能可以作为方向被激活和组合，以实现更高级别的功能和个人化优化。

Abstract

Skills, as a useful abstraction for the procedural capabilities of large language models (LLMs), capture how models perform structured, multi-step reasoning and program execution. Existing approaches typically treat skills as explicit, surface-level constructs specified through prompts or programs, leaving open the question of how such procedural capabilities are represented inside the model and whether they can be manipulated as structured objects in latent space. In this empirical study, we investigate whether procedural LLM skills can be represented as directions in activation space and whether vector-space operations over these directions can express skill-level behaviors. We find that procedural skills admit a vector-space representation: individual skill directions can be activated to shift model behavior; independently extracted directions can compose to form higher-level skills. Contrastive directions yield context-conditioned algorithmic personalization and optimization trajectories over skill directions evolve non-monotonically, with intermediate states often surpassing fully optimized solutions. These results support a representation-level view of procedural LLM skills: they admit a latent vector-space organization that allows direct manipulation through internal interventions.

Problem

• Latent Space
• Procedural Skills
• Activation Space
• Vector-Space Representation
• Large Language Models

Innovation

• latent vector-space representation
• procedural skills
• activation space
• skill-level behaviors
• algorithmic personalization