Search papers, labs, and topics across Lattice.
This study explores parameter-efficient fine-tuning (PEFT) methods, specifically adapters and Low-Rank Adaptation (LoRA), for transformer-based models in instance segmentation tasks. By fine-tuning only 1-6% of model parameters, the research achieves competitive performance compared to traditional methods that require 40-55%, highlighting the efficiency of using 2-3 adapters per transformer block. The findings reveal that the effectiveness of PEFT techniques is context-dependent, varying with dataset complexity and model architecture, thus paving the way for scalable transfer learning solutions.
Fine-tuning just 1-6% of parameters can match traditional methods, revolutionizing efficiency in instance segmentation tasks.
Research and applications in artificial intelligence have recently shifted with the rise of large pretrained models, which deliver state-of-the-art results across numerous tasks. However, the substantial increase in parameters introduces a need for parameter-efficient training strategies. Despite significant advancements, limited research has explored parameter-efficient fine-tuning (PEFT) methods in the context of transformer-based models for instance segmentation. Addressing this gap, this study investigates the effectiveness of PEFT methods, specifically adapters and Low-Rank Adaptation (LoRA), applied to two models across four benchmark datasets. Integrating sequentially arranged adapter modules and applying LoRA to deformable attention--explored here for the first time--achieves competitive performance while fine-tuning only about 1-6% of model parameters, a marked improvement over the 40-55% required in traditional fine-tuning. Key findings indicate that using 2-3 adapters per transformer block offers an optimal balance of performance and efficiency. Furthermore, LoRA, exhibits strong parameter efficiency when applied to deformable attention, and in certain cases surpasses adapter configurations. These results show that the impact of PEFT techniques varies based on dataset complexity and model architecture, underscoring the importance of context-specific tuning. Overall, this work demonstrates the potential of PEFT to enable scalable, customizable, and computationally efficient transfer learning for instance segmentation tasks.
