Parameter-Efficient Adapter Tuning for Tabular-Image Multimodal Learning

2026-06-10

AI Summary

This work addresses the high computational cost of full fine-tuning and the limited adaptability of frozen encoders in table–image multimodal learning. To this end, the authors propose TI-Adapter, a parameter-efficient framework that freezes pretrained encoders while introducing lightweight, modality-specific adapters—comprising dedicated embedding and bottleneck layers for tables and images, respectively. The study innovatively designs distinct adapter architectures for each modality and systematically investigates the trade-offs between performance and efficiency when inserting these adapters at different network positions. Extensive experiments across 20 table–image datasets demonstrate that TI-Adapter achieves performance on par with or even superior to full fine-tuning, using only a minimal number of trainable parameters.

Abstract

Tabular-image multimodal learning aims to improve predictive modeling by jointly using structured tabular attributes and visual data. Although pretrained encoders provide strong modality-specific representations, full fine-tuning can be computationally expensive, while keeping encoders frozen may limit task-specific adaptation. We propose the Tabular-Image Adapter (TI-Adapter), a modality-specific adapter-based fine-tuning framework for efficient multimodal adaptation. TI-Adapter freezes the pretrained tabular encoder and learns an adapter after the extracted tabular embedding, while adapting the image branch with embedding-level and bottleneck-level adapters instead of full fine-tuning. Experiments on 20 tabular-image datasets show that TI-Adapter achieves competitive or better predictive performance than full fine-tuning while using substantially fewer trainable parameters. Ablation studies further demonstrate the importance of adapter placement for balancing performance and practical efficiency.

Problem

• tabular-image multimodal learning
• parameter-efficient tuning
• adapter-based fine-tuning
• pretrained encoders
• task-specific adaptation

Innovation

• parameter-efficient tuning
• adapter
• tabular-image multimodal learning
• frozen pretrained encoders
• multimodal adaptation