Scalable Semantic Steering of Embedding Projections

2026-07-04

AI Summary

This work addresses the scalability limitations of existing semantic-guided dimensionality reduction methods, which incur linearly growing computational costs due to per-sample invocation of large language models (LLMs). To overcome this, the authors propose a group-level semantic prototyping framework that shifts semantic computation from individual samples to user-defined groups. A single LLM call generates a structured group profile, which—combined with seed centroids—forms hybrid semantic prototypes. Efficient semantic alignment in the embedding space is then achieved through soft assignment, an abstention mechanism, and alignment-aware scaling updates. Evaluated on LitCovid (5K documents), the method reduces LLM invocations by over three orders of magnitude while matching the global alignment performance of per-sample approaches. Its multimodal applicability is further demonstrated on image data.

Abstract

Low-dimensional projections support interactive visual analysis of high-dimensional data embeddings, but their structure often does not align with analyst-defined semantic relationships. Recent LLM-augmented semantic steering methods address this gap by externalizing analyst intent from user-defined groups of seed examples, but they propagate intent through per-item LLM reasoning, causing LLM calls and cost to grow linearly with collection size. We propose a scalable semantic steering method that shifts semantic computation from individual items to user-defined groups. A single LLM call generates structured profiles for all groups, which are embedded and combined with seed centroids to form hybrid semantic prototypes. The method then propagates intent without retraining, using embedding-space soft assignment, abstention, and alignment-scaled updates before reprojection. On a 5K-document LitCovid corpus, our method achieves global alignment comparable to per-item LLM steering while reducing LLM calls by over three orders of magnitude. An image case study shows that the same prototype-based mechanism extends to multimodal embeddings. These results suggest that group-level representations can make semantic steering more practical for larger embedding collections.

Problem

• semantic steering
• embedding projections
• scalability
• LLM-augmented methods
• high-dimensional data

Innovation

• semantic steering
• scalable LLM integration
• embedding projection
• group-level prototypes
• multimodal embeddings