Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
The authors introduce MasterSet, a large-scale benchmark for must-cite citation recommendation in AI/ML, addressing the increasing difficulty of comprehensive citation coverage. They annotate over 150,000 papers from 15 top venues with a three-tier labeling scheme, using an LLM-based judge validated by human experts. Experiments with sparse retrieval, dense embeddings, and graph-based methods show that retrieving must-cite papers remains a challenging task.
Finding the "must-cite" papers in AI/ML is harder than you think, and this new benchmark with 150,000 papers proves it.
The explosive growth of AI and machine learning literature -- with venues like NeurIPS and ICLR now accepting thousands of papers annually -- has made comprehensive citation coverage increasingly difficult for researchers. While citation recommendation has been studied for over a decade, existing systems primarily focus on broad relevance rather than identifying the critical set of ``must-cite''papers: direct experimental baselines, foundational methods, and core dependencies whose omission would misrepresent a contribution's novelty or undermine reproducibility. We introduce MasterSet, a large-scale benchmark specifically designed to evaluate must-cite recommendation in the AI/ML domain. MasterSet incorporates over 150,000 papers collected from official conference proceedings/websites of 15 leading venues, serving as a comprehensive candidate pool for retrieval. We annotate citations with a three-tier labeling scheme: (I) experimental baseline status, (II) core relevance (1--5 scale), and (III) intra-paper mention frequency. Our annotation pipeline leverages an LLM-based judge, validated by human experts on a stratified sample. The benchmark task requires retrieving must-cite papers from the candidate pool given only a query paper's title and abstract, evaluated by Recall@$K$. We establish baselines using sparse retrieval, dense scientific embeddings, and graph-based methods, demonstrating that must-cite retrieval remains a challenging open problem.