Search papers, labs, and topics across Lattice.
The paper introduces ADAMAB, a sample-efficient embedding calibration framework for few-shot implicit pattern recognition that trains lightweight, embedder-agnostic calibrators on top of fixed embedding models. To address data scarcity, ADAMAB employs an adaptive data augmentation strategy guided by a Multi-Armed Bandit (MAB) mechanism with a modified upper confidence bound algorithm to mitigate gradient shifting. Experiments demonstrate that ADAMAB achieves up to 40% accuracy improvement with fewer than 5 initial data samples per class compared to existing methods.
Achieve up to 40% accuracy gains in few-shot pattern recognition by calibrating embeddings with a multi-armed bandit-driven data augmentation strategy.
Recognizing implicit visual and textual patterns is essential in many real-world applications of modern AI. However, tackling long-tail pattern recognition tasks remains challenging for current pre-trained foundation models such as LLMs and VLMs. While finetuning pre-trained models can improve accuracy in recognizing implicit patterns, it is usually infeasible due to a lack of training data and high computational overhead. In this paper, we propose ADAMAB, an efficient embedding calibration framework for few-shot pattern recognition. To maximally reduce the computational costs, ADAMAB trains embedder-agnostic light-weight calibrators on top of fixed embedding models without accessing their parameters. To mitigate the need for large-scale training data, we introduce an adaptive data augmentation strategy based on the Multi-Armed Bandit (MAB) mechanism. With a modified upper confidence bound algorithm, ADAMAB diminishes the gradient shifting and offers theoretically guaranteed convergence in few-shot training. Our multi-modal experiments justify the superior performance of ADAMAB, with up to 40% accuracy improvement when training with less than 5 initial data samples of each class.
