Search papers, labs, and topics across Lattice.
This paper introduces the CUP dataset, a benchmark for evaluating book retrieval methods in the Greek language, comprising 868 catalog records and 104 expert-annotated queries. The study compares various retrieval techniques, including sparse, dense, hybrid, and LLM-assisted methods, revealing that multilingual embeddings outperform Greek-specific models while hybrid retrieval yields the best overall performance. Notably, the analysis highlights that BM25 is particularly effective for named-entity queries, whereas dense and hybrid methods excel in handling natural-language and noisy queries, underscoring the nuanced performance of different approaches in real-world retrieval scenarios.
Multilingual embeddings outperform Greek-specific models in book retrieval, revealing surprising strengths in hybrid methods for complex queries.
We present CUP, a Greek book retrieval benchmark consisting of 868 catalog records and 104 expert-annotated queries with graded relevance judgments. We evaluate sparse (BM25), dense (sentence-transformers), hybrid, and LLM-assisted retrieval methods in this book-search setting. Multilingual embeddings outperform Greek-specific models, while hybrid retrieval performs best overall. A query-level analysis shows that BM25 excels at named-entity queries, while dense and hybrid methods improve natural-language, noisy, cross-lingual, and concept queries. Field-aware prompting has model-specific effects, while LLM TOC summarization improves TOC-only retrieval and LLM post-filtering improves early-stage retrieval at a high cost. Overall, CUP enables real-world evaluation of Greek retrieval across lexical, semantic, noisy, and cross-lingual queries.
