Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This paper investigates the label complexity required for certifying that the missed relevant items in a high-recall candidate generation stage are minimal, highlighting that effective auditing must sample from the excluded pool rather than relying solely on the candidate set. The authors establish a minimax rate-optimal bound for the number of labels needed to ensure that the missed mass is small, proving that any valid audit must inspect a significant number of excluded-pool labels. Additionally, they introduce a finite-sample toolkit that allows for precise certification of missed mass and recall while accommodating various candidate generator families and perturbation mechanisms.
Certifying missed relevant items in high-recall candidate generation requires sampling from the excluded pool, revealing a surprising minimax rate-optimal label complexity.
An initial high-recall stage in an empirical pipeline decides which items pass to later review, labelling, or modelling, and relevant items it misses are lost to every subsequent stage. We study how many audit labels are needed to certify, with finite-sample validity, that this missed relevant mass is small, and our main results characterise the label complexity of this problem. We first show that no procedure using only labels from inside the candidate set can certify any non-trivial bound on the missed mass: the audit must sample the excluded pool, the only region where unrecovered relevant items can lie. We then prove a matching finite-corpus lower bound. Any valid audit that certifies fewer than $m$ missed relevant items with high probability when none are present, even if adaptive and permitted to label the entire included pool, must inspect on the order of $N_0/m$ excluded-pool labels. Excluded-pool auditing is therefore minimax rate-optimal, not merely convenient, for missed-mass certification in the zero-miss regime. Building on this characterisation, we develop an exact finite-sample toolkit, using binomial and hypergeometric inversion rather than asymptotic approximation, that certifies missed mass, converts it to recall through a two-pool design, certifies pre-specified families of nested candidate generators simultaneously, and produces stress-test certificates against declared perturbation mechanisms. These certificates can be paired with observable review burden to select the least burdensome pre-specified candidate generator meeting a missed-mass target. Every guarantee holds under one discipline: the candidate generator, or the pre-specified family from which it is selected, and the audit rule are fixed before the certification labels are examined.