Vision Non-Causal Trapezoidal Mamba: Eliminating Directional Scanning in Vision SSMs with Second-Order Dynamics

2026-07-03

AI Summary

This work addresses the spatial bias and orientation sensitivity inherent in existing vision state space models (SSMs), which rely on causal directional scanning. To overcome these limitations, the authors propose VNCT—the first second-order non-causal visual SSM—that eliminates directional scanning entirely and enables fully parallel, global token interactions across the entire image in a single forward pass. Built upon a non-causal Trapezoidal Mamba architecture and incorporating second-order dynamic modeling, VNCT significantly enhances robustness to geometric transformations such as rotation and flipping. The model achieves state-of-the-art performance across ImageNet-1K, COCO, and ADE20K benchmarks, surpassing both directional and first-order non-causal SSMs, with notable gains including up to a 3.7-point improvement in boundary IoU and new records in both classification and segmentation tasks.

Abstract

State Space Models (SSMs) have emerged as an alternative to Vision Transformers, yet most vision SSMs inherit directional token scanning from causal sequence modeling. While effective for sequential data, directional scanning introduces spatial bias and orientation-sensitive representations. We present Vision Non-Causal Trapezoidal Mamba (VNCT), a second-order non-causal vision SSM that enables all image tokens to interact in a single pass, eliminating direSctional scanning and achieving low single-image inference latency. VNCT exhibits more orientation-robust representations, showing reduced performance degradation under image rotations and flips, while improving Boundary IoU by up to 3.7 points, leading to more accurate boundary preservation and object localization. Across ImageNet-1K classification, COCO object detection and instance segmentation, and ADE20K semantic segmentation, VNCT consistently outperforms both directional-scanning vision SSMs and first-order non-causal SSMs. These results show that directional scanning is unnecessary for high-performance vision SSMs and that second-order non-causal state-space modeling offers a simple, efficient, and robust alternative for visual recognition.

Problem

• directional scanning
• spatial bias
• orientation sensitivity
• vision state space models
• image representation robustness

Innovation

• non-causal
• second-order dynamics
• state space models
• directional scanning
• vision SSMs