Search papers, labs, and topics across Lattice.
This paper introduces the Tabular-Image Adapter (TI-Adapter), a novel framework for parameter-efficient fine-tuning in tabular-image multimodal learning that combines structured tabular data with visual inputs. By freezing the pretrained tabular encoder and employing modality-specific adapters for the image branch, TI-Adapter significantly reduces the number of trainable parameters while maintaining or improving predictive performance across 20 datasets. The findings highlight the critical role of adapter placement in optimizing both efficiency and effectiveness in multimodal tasks.
Achieving competitive predictive performance with significantly fewer parameters, TI-Adapter redefines efficiency in multimodal learning.
Tabular-image multimodal learning aims to improve predictive modeling by jointly using structured tabular attributes and visual data. Although pretrained encoders provide strong modality-specific representations, full fine-tuning can be computationally expensive, while keeping encoders frozen may limit task-specific adaptation. We propose the Tabular-Image Adapter (TI-Adapter), a modality-specific adapter-based fine-tuning framework for efficient multimodal adaptation. TI-Adapter freezes the pretrained tabular encoder and learns an adapter after the extracted tabular embedding, while adapting the image branch with embedding-level and bottleneck-level adapters instead of full fine-tuning. Experiments on 20 tabular-image datasets show that TI-Adapter achieves competitive or better predictive performance than full fine-tuning while using substantially fewer trainable parameters. Ablation studies further demonstrate the importance of adapter placement for balancing performance and practical efficiency.
