Search papers, labs, and topics across Lattice.
This study evaluates a fixed retrieval-augmented generation (RAG) pipeline for scientific question answering using the CORD-19 dataset, examining its performance across three different corpus sizes. The results reveal that hybrid retrieval methods significantly outperform sparse-only or dense-only approaches, achieving a perfect Recall@10 at both the 1K and 15K chunk scales. Notably, the use of an MS MARCO-trained cross-encoder reranker was found to decrease precision due to domain mismatch, highlighting the importance of aligning models with specific scientific contexts.
Hybrid retrieval methods can achieve perfect recall in scientific question answering, but domain mismatch can undermine precision when using general-purpose rerankers.
We introduce SciRet, a compute-aware empirical study of retrieval-augmented generation for scientific question answering over CORD-19. Rather than proposing a new model, we evaluate a fixed scientific RAG pipeline across three corpus scales: 1,034 chunks (1K papers), 5,160 chunks (5K papers), and 15,480 chunks (15K papers). The pipeline combines sentence-window chunking, BM25, BGE-M3 dense retrieval, reciprocal rank fusion, optional cross-encoder reranking, and grounded answer generation. Across these settings, hybrid retrieval is more robust than either sparse-only or dense-only retrieval in our setting, reaching Recall@10 of 1.000 at 1K and 15K. In contrast, an MS MARCO-trained cross-encoder reranker reduces precision on the scientific corpus, suggesting that domain mismatch can outweigh the benefits of stronger query-passage interaction. Generation faithfulness measured with RAGAS increases with corpus scale in our setup. Retrieval evaluation uses pseudo-relevance labels derived from the hybrid system, so we treat the results as controlled comparative evidence rather than a benchmark claim. We release code, indexes, and evaluation outputs to support replication and follow-up studies.
