Forewarned is Forearmed: When Non-Sequential Embedding Turns Into an Anomaly Detector

2026-06-29

AI Summary

This work addresses the vulnerability of non-sequential multimodal sentence embeddings—such as those produced by SONAR—to perturbations during decoding, which compromises representation reliability. The study is the first to uncover a strong association between specific embedding-space dimensions and decoding anomalies. Building on this insight, the authors propose an anomaly detection method grounded in encoder-decoder consistency and introduce targeted interventions on sensitive dimensions to correct these irregularities. The approach substantially enhances the robustness of multimodal representations and demonstrates the efficacy of dimension-level correction, offering a novel pathway toward improving the reliability of non-sequential embeddings.

Abstract

This paper offers an in-depth analysis of non-sequential multimodal sentence-level embeddings, with a particular focus on the SONAR model. We demonstrate that certain embedding dimensions are sensitive to perturbations and can serve as indicators of decoding anomalies. By leveraging the consistency between successive encoding and decoding, we successfully build an accurate detector. Additionally, we explore modifying specific dimensions of interest to attempt to correct them. This work underscores the importance of understanding and analyzing the embeddings themselves to enhance the reliability of multimodal representations.

Problem

• non-sequential embedding
• anomaly detection
• multimodal representation
• decoding anomaly
• embedding sensitivity

Innovation

• non-sequential embedding
• anomaly detection
• multimodal representation
• SONAR model
• embedding perturbation