Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper addresses the challenge of multi-target estimation in large image collections, where traditional methods struggle with bias and lack statistical guarantees. By adapting sampling and estimation strategies from survey sampling, the authors demonstrate that importance sampling is effective for moderate annotation budgets, while uniform sampling with control variates excels in scenarios with many targets and minimal labels. Their framework successfully integrates biased model predictions with limited human annotations, providing rigorous and reliable population-level estimates across various datasets.
Importance sampling can significantly enhance the accuracy of multi-target estimates in large image collections, outperforming traditional methods under specific conditions.
Computer vision models are increasingly used as measurement tools to estimate population-level quantities from large image collections, but prediction errors introduce bias and the resulting estimates lack statistical guarantees required in scientific applications. Prior work uses a Monte Carlo framework to combine model predictions with ground-truth annotations by sampling some images for humans to label and is able to provide unbiased estimates with controllable accuracy, but primarily addresses single-scalar estimation. We study the more general problem of multi-target estimation, where many quantities (e.g., class counts or proportions) must be estimated simultaneously, and adapt sampling and estimation strategies from survey sampling to this setting. Evaluations on five detection and segmentation datasets with 7-80 classes show that importance sampling excels with moderate annotation budgets or fewer targets, whereas uniform sampling with control variates is superior when estimating many targets or operating with minimal labels. Additionally, a subset-based ratio estimator remains highly competitive across all regimes. Ultimately, our framework effectively combines biased model predictions and limited human labels into rigorous scientific measurements.