Problem
Research questions and friction points this paper is trying to address.
token-level text anomaly detection
global distance calculation
local anomaly signals
pre-trained language models
over-smoothing effect
SIM: Subspace Interaction-based Method for Token-Level Text Anomaly Detection
🤖 AI Summary
针对文本异常检测中局部信号稀释和过平滑问题，提出基于子空间交互的方法SIM，通过分解词嵌入维度并生成伪异常来增强局部异常信号。
📝 Abstract
Token-level text anomaly detection, as an emerging trend of text anomaly detection, moves beyond coarse-grained document-level detection by localizing anomalous tokens within text. By providing fine-grained abnormality prediction, token-level text anomaly detection plays a critical role in various real-world applications, such as spam filtering and fake news detection. However, existing methods still rely on the global distance calculation for scoring, during which the local anomaly signals are severely diluted by numerous redundant normal feature dimensions. Moreover, pre-trained language models used in these methods inevitably smooth out surface anomalies, further limiting their effectiveness in token-level anomaly detection. To address these limitations, we propose a Subspace Interaction-based Method (SIM for short) for token-level text anomaly detection. To prevent local signal dilution, SIM adopts a subspace interaction-based anomaly detector, which decouples high-dimensional token embeddings into multiple low-dimensional ones, amplifying localized anomaly signals hidden within specific dimensions. To counteract the over-smoothing effect, we design a hard pseudo-anomaly generation module to construct pseudo-anomalous tokens, simulating the subtle anomalies obscured by semantic smoothing. Also, a probabilistic boundary loss is developed to standardize anomaly scores into statistical distances, effectively enforcing anomalous instances to deviate significantly from the normal distribution center. Extensive experiments on multiple benchmark datasets verify the effectiveness of SIM and demonstrate its remarkable efficiency, robustness, and interpretability. The source code is available at: https://github.com/yankehan/SIM-TAD.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Subspace Interaction
Token-Level Anomaly Detection
Hard Pseudo-Anomaly Generation
Probabilistic Boundary Loss
🔎 Similar Papers
No similar papers found.
