HH-SAE: Discovering and Steering Hierarchical Knowledge of Complex Manifolds

2026-05-11

AI Summary

This study addresses the feature density conflict in high-dimensional critical tasks, where rare semantic innovations are obscured by dense background contexts. To resolve this, the authors propose a Hybrid Hierarchical Sparse Autoencoder (HH-SAE), which introduces—for the first time—a three-tier nested manifold decomposition mechanism encompassing contextual (L₀), atomic (f₁), and compositional (f₂) layers. By integrating context stripping, knowledge-guided synthesis, and path ablation validation, the model prioritizes capturing high-order mechanistic innovations over spurious environmental proxies. Evaluated on cross-domain zero-shot fraud detection, HH-SAE achieves an AUC of 0.9156. Knowledge-guided synthesis improves AUPRC by 9.9% over the state of the art, while ablating the contextual layer causes a 13.46% performance drop, confirming both the efficacy and necessity of the proposed architecture.

Abstract

Rare semantic innovations in high-dimensional, mission-critical domains are often obscured by dense background contexts, a challenge we define as \textit{feature density conflict}. We introduce the \textbf{Hybrid Hierarchical SAE (HH-SAE)} to resolve this by factorizing manifolds into a nested hierarchy of \textbf{Contextual} ($L_0$), \textbf{Atomic} ($f_1$), and \textbf{Compository} ($f_2$) tiers. Evaluating across disparate manifolds, HH-SAE demonstrates superior resolution by \textbf{``fracturing'' administrative clinical labels into physiological modes} and achieving a peak \textbf{cross-domain zero-shot AUC of 0.9156 in fraud detection}. Path ablation confirms the architecture's structural necessity, revealing a 13.46\% utility collapse when contextual subtraction is removed. Finally, knowledge-steered synthesis achieves a +9.9\% AUPRC lift over state-of-the-art generators, proving that HH-SAE effectively prioritizes high-order mechanistic innovation over environmental proxies to enable high-precision discovery in high-stakes environments.

Problem

• feature density conflict
• semantic innovation
• high-dimensional manifolds
• contextual obscuration
• mission-critical domains

Innovation

• Hybrid Hierarchical SAE
• feature density conflict
• manifold factorization
• zero-shot fraud detection
• knowledge-steered synthesis