Search papers, labs, and topics across Lattice.
This paper introduces VeriCam, a novel pipeline that enhances zero-shot classification by learning specialized features for classifying unknown data. By leveraging the representational power of image models trained for verification tasks, VeriCam constructs a relational graph that captures intricate class relationships, enabling fine-grained classification in challenging real-world scenarios. The approach is validated on the LPLCv2 dataset, achieving an F1-Score of 93.45 and a V-Measure score of 80.13, demonstrating its effectiveness in overcoming biases inherent in traffic surveillance data.
Achieving a remarkable F1-Score of 93.45, VeriCam redefines how we tackle zero-shot classification of unknown classes in biased real-world datasets.
The advent of foundation models have enabled a new era in zero-shot classification. Yet, key challenges persist. Despite their impressive generalization power that leverages the immense pre-training knowledge, both foundation models for image and text as well as vision-text hybrids lack the representational power needed for fine-grained, minutiae-based class separation that some real-world tasks require. To address the current gaps in the literature, we propose VeriCam, a pipeline designed to learn highly specialized features that enable classification of unknown classes in unseen data. VeriCam works by leveraging the representation power of image models trained for the verification task, where the model develops an intricate feature space that incorporates fine-grained details. By training a model to discriminate between pairs of images from the same and different classes, a relational graph is constructed, representing the class relationships between data points. We then present two approaches for graph clustering: a naive algorithm and a specific setup for the Leiden graph clustering algorithm. The pipeline is validated on the LPLCv2 dataset, which comprises real-world traffic surveillance images. We show that the dataset carries an inherent capture device bias that is posed as a generalization challenge for downstream License Plate recognition tasks such as OCR. As such, we dynamically identify capture devices with a label-agnostic approach, enabling the construction of a fair and unbiased benchmark. In the cross-device scenario, our pipeline reaches an F1-Score of 93.45 in the verification baseline and a V-Measure score of 80.13 in the clustering step. All code is publicly available at https://github.com/lmlwojcik/VeriCam
