Search papers, labs, and topics across Lattice.
This paper introduces RATIO, a large-scale benchmark designed to enhance retrieval in scientific literature by defining relevance through three ideation moves: Address, Broaden, and Specify. By leveraging millions of full-text scientific papers and employing a novel approach that combines discourse-marker distant supervision with extensive vetting from both LLMs and humans, the authors demonstrate that operation-specific fine-tuning significantly improves retrieval performance. The findings highlight the potential for RATIO to serve as a foundational framework for advancing literature-grounded ideation in AI and human scientific inquiry.
Retrieval methods can be dramatically improved by tailoring them to specific ideation operations, as shown by RATIO's performance enhancements.
Retrieved scientific literature can serve as inspiration for both human and AI scientists. Inspiration can take different forms: prior work may directly suggest how to address a problem, or surface directions at different levels of abstraction - zooming out to a more general view or zooming in to a concrete realization. We introduce RATIO (Retrieval Across Typed Ideation Operations), a large-scale benchmark in which relevance is defined by three operations which we name ideation moves: Address retrieves potential approaches for stated problems, Broaden retrieves more general formulations, and Specify retrieves concrete instantiations. RATIO is constructed from millions of full-text scientific papers across CS literature via a general recipe that extends discourse-marker distant supervision - previously used only for classification - to corpus-scale retrieval, combined with extensive LLM and human vetting. Experiments show that operation-specific fine-tuning substantially boosts retrievers but leaves much room for further improvements. RATIO provides a scalable training and evaluation framework for retrieval components that support literature-grounded ideation, opening up new research avenues on scientific inspiration retrieval.
