Tydra: An Efficient Hybrid Model for Tabular Data

2026-08-21

AI Summary

为解决表格数据预测中准确性和效率的平衡问题，提出Tydra模型，结合Transformer和状态空间模型，减少推理时间同时保持较高预测性能。

Abstract

Transformer-based tabular foundation models such as TabPFN achieve strong predictive performance but incur quadratic computational cost with context length. On the other hand, subquadratic SSM-based alternatives such as Hydra trade away accuracy for efficiency. To balance both, we introduce Tydra, a hybrid Transformer-State Space Model (SSM) architecture for tabular in-context learning that interleaves attention and SSM layers. Across 30 OpenML datasets, Tydra reduces inference time by 30% relative to TabPFN while retaining much of its predictive performance. Tydra also outperforms an approximately ten-times-larger Hydra model while providing faster inference. The results indicate that hybrid architectures are a promising direction for tabular foundation models.

Problem

• Transformer
• Tabular Data
• Computational Cost
• Efficiency
• Accuracy

Innovation

• Hybrid Architecture
• Transformer-SSM
• In-context Learning
• Efficiency-Accuracy Trade-off