Semantic Sections: An Atlas-Native Feature Ontology for Obstructed Representation Spaces

2026-03-21

AI Summary

Traditional feature representations struggle to capture semantic structures that are locally consistent yet globally inconsistent within obstructed representation spaces. This work proposes “semantic sections” as a novel feature ontology, modeling semantics through families of local features defined over contextual atlases and their transport compatibility. It distinguishes three types of sections—locally extendable, globally extendable, and twisted—and introduces a globalization criterion based on cycle consistency, thereby transcending conventional assumptions of global feature coherence. By integrating seed propagation, overlapping synchronization, defect pruning, and cycle-aware classification, the authors establish a complete pipeline for section discovery and verification. Extensive semantic sections are identified in the 16th layer of multiple large language models, significantly outperforming baseline methods based on raw vector similarity in identity recovery tasks, thus demonstrating the approach’s effectiveness and superiority.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Local Search
• Machine Learning: Representation Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs

Abstract

Recent interpretability work often treats a feature as a single global direction, dictionary atom, or latent coordinate shared across contexts. We argue that this ontology can fail in obstructed representation spaces, where locally coherent meanings need not assemble into one globally consistent feature. We introduce an atlas-native replacement object, the semantic section: a transport-compatible family of local feature representatives defined over a context atlas. We formalize semantic sections, prove that tree-supported propagation is always pathwise realizable, and show that cycle consistency is the key criterion for genuine globalization. This yields a distinction between tree-local, globalizable, and twisted sections, with twisted sections capturing locally coherent but holonomy-obstructed meanings. We then develop a discovery-and-certification pipeline based on seeded propagation, synchronization across overlaps, defect-based pruning, cycle-aware taxonomy, and deduplication. Across layer-16 atlases for Llama 3.2 3B Instruct, Qwen 2.5 3B Instruct, and Gemma 2 2B IT, we find nontrivial populations of semantic sections, including cycle-supported globalizable and twisted regimes after deduplication. Most importantly, semantic identity is not recovered by raw global-vector similarity. Even certified globalizable sections show low cross-chart signed cosine similarity, and raw similarity baselines recover only a small fraction of true within-section pairs, often collapsing at moderate thresholds. By contrast, section-based identity recovery is perfect on certified supports. These results support semantic sections as a better feature ontology in obstructed regimes.

Problem

• obstructed representation spaces
• feature ontology
• semantic sections
• global consistency
• local coherence

Innovation

• semantic section
• atlas-native representation
• cycle consistency
• feature ontology
• obstructed representation space