Search papers, labs, and topics across Lattice.
This paper introduces the Conditional-Composition Domain Match (CCDM) metric family, designed to evaluate the effectiveness of synthetic datasets for object detection without the need for extensive training. By leveraging this pre-computable metric, researchers can efficiently assess the utility of synthetic training sets, addressing the computational burden associated with traditional evaluation methods. Experiments demonstrate that CCDM achieves a perfect Spearman correlation of 1.0 with YOLOv8's downstream performance on the VisDrone-DET dataset, significantly surpassing existing evaluation metrics.
CCDM offers a groundbreaking way to predict synthetic dataset effectiveness, achieving perfect correlation with model performance without the need for training.
With the recent advent of image generative models, synthetic data are increasingly being used to supplement limited real datasets for training computer vision models. However, not all synthetic datasets improve performance equally, and their effectiveness can only be assessed by training a downstream model, which is computationally expensive and time-consuming. This problem is pronounced in the task of object detection, where the required annotations are much more dense due to bounding boxes. In this paper, we propose a pre-computable metric family, dubbed Conditional-Composition Domain Match (CCDM), which serves as a proxy for the relative utility of candidate synthetic training sets for downstream detection. Experiments on the VisDrone-DET dataset show that the CCDM metric families achieve a Spearman correlation of 1.0 with the downstream performance of YOLOv8, clearly outperforming existing metrics for synthetic image evaluation.
