Unlocking the Power of Medical Tabular Data via Semantic-Aware Multimodal Pre-training

2026-08-11

AI Summary

This work addresses a critical limitation in existing medical multimodal pretraining approaches, which often overlook the dense quantitative diagnostic phenotypes embedded in structured clinical tables and suffer from restricted representational capacity due to semantically blind designs. To overcome this, the authors propose a semantic-aware multimodal pretraining framework that explicitly models the two-dimensional intrinsic structure of medical tabular data. The framework introduces an importance-aware adaptive masking strategy to construct an unsupervised curriculum learning scheme and incorporates a soft-label discretization module that replaces unstable numerical regression with stable distribution matching, thereby preserving ordinal relationships effectively. By unifying vision, language, and structured tabular modalities, the method achieves new state-of-the-art performance across multiple large-scale medical benchmarks—including SLICE-3D, HOP, and EyePACS—demonstrating significantly enhanced model robustness and cross-domain generalization.

Abstract

While vision-language models dominate medical representation learning, unstructured text lacks the dense, quantitative diagnostic phenotypes inherent in structured clinical tables. However, existing multimodal pre-training methods underutilize this potential due to semantic-agnostic designs that treat tabular inputs as flat vectors and employ unstable continuous regression objectives. To overcome this, we propose a novel semantic-aware framework explicitly modeling the intrinsic two-dimensional structure of tabular data. First, addressing the inter-feature hierarchy of varying diagnostic importance, we introduce Importance-Aware Adaptive Masking to construct a label-free curriculum prioritizing salient features. Second, addressing the intra-feature continuity-discreteness duality, we propose a Soft-Label Discretized Module that replaces unstable numerical regression with stable distribution matching, thereby mathematically preserving ordinal relationships. Extensive experiments across large-scale dermatology (SLICE-3D, HOP) and ophthalmology (EyePACS) datasets establish a new state-of-the-art (SOTA), demonstrating exceptional robustness and cross-domain generalizability.

Problem

• medical tabular data
• multimodal pre-training
• semantic-aware
• structured clinical tables
• diagnostic phenotypes

Innovation

• semantic-aware pre-training
• tabular data modeling
• importance-aware masking
• soft-label discretization
• multimodal representation learning