DynaTab: Dynamic Feature Ordering as Neural Rewiring for High-Dimensional Tabular Data

2026-05-05

AI Summary

High-dimensional tabular data lacks a natural feature ordering, which limits the applicability of sequence-sensitive deep models. This work proposes DynaTab, a novel end-to-end trainable architecture that introduces a dynamic feature reordering paradigm. DynaTab employs a lightweight complexity-based criterion to predict the potential gain from reordering, then dynamically adjusts feature sequences via a neural rewiring mechanism. It further integrates learnable positional embeddings, importance gating, and masked attention to seamlessly adapt to any sequence-sensitive backbone network. Evaluated across 36 real-world high-dimensional tabular datasets, DynaTab consistently outperforms 45 state-of-the-art baselines, demonstrating particularly strong performance in high-dimensional settings.

Abstract

High-dimensional tabular data lacks a natural feature order, limiting the applicability of permutation-sensitive deep learning models. We propose DynaTab, a dynamic feature ordering-enabled architecture inspired by neural rewiring. We introduce a lightweight criterion that predicts when feature permutation will benefit a dataset by quantifying its intrinsic complexity. DynaTab dynamically reorders features via a neural rewiring algorithm and processes them through a compact, dynamic order-aware combination of separate learned positional embedding, importance-based gating, and masked attention layers, compatible with any sequence-sensitive backbone. Trained end-to-end with bespoke dynamic feature ordering (DFO) and dispersion losses, DynaTab achieves statistically significant gains, particularly on high-dimensional datasets, where it is benchmarked against 45 state-of-the-art baselines across 36 different real-world tabular datasets. Our results position DynaTab as a compelling new paradigm for high-dimensional tabular deep learning.

Problem

• high-dimensional tabular data
• feature ordering
• permutation-sensitive models
• neural rewiring
• deep learning

Innovation

• dynamic feature ordering
• neural rewiring
• tabular deep learning
• positional embedding
• masked attention