Search papers, labs, and topics across Lattice.
This paper investigates the effectiveness of various retrieval strategies for answering questions in Arabic Islamic jurisprudence (fiqh), focusing on the distinction between relevant answer-bearing passages and similar but non-relevant content. By constructing a dedicated retrieval test collection, the authors evaluate dense, lexical, hybrid, and fine-tuned retrieval methods, revealing that fine-tuning significantly enhances performance, while madhhab-aware filtering notably improves retrieval accuracy for school-specific queries. The best-performing retriever achieves a mean reciprocal rank (MRR) of 0.553, highlighting the critical importance of precise retrieval in the context of Islamic jurisprudence.
Madhhab-aware filtering can more than double retrieval accuracy for school-specific fiqh questions, revealing a crucial dimension in Islamic jurisprudence retrieval.
Retrieval-Augmented Generation is used for Islamic question answering, but most systems are evaluated end-to-end, making retrieval failures difficult to isolate from generation failures. We study answer-bearing retrieval for Arabic fiqh, where a passage is relevant only if it states the ruling required by the question. We build a retrieval test collection for Arabic fiqh and use it to evaluate dense, lexical, hybrid, fine-tuned, and madhhab-aware retrieval strategies. The best retriever achieves 0.524 MRR@5, while fine-tuning improves performance to 0.553. Hybrid retrieval provides limited gains for strong models, whereas madhhab-aware filtering more than doubles MRR@5 on school-specific questions. We further present an error analysis showing that the main challenge is distinguishing answer-bearing passages from topically similar passages that do not contain the requested ruling.
