Search papers, labs, and topics across Lattice.
This paper introduces RAQG-QPP, a novel Query Performance Prediction (QPP) method that leverages retrieved query variants (QVs) from a query log and retrieval-augmented query generation using LLMs. By conditioning LLM-generated QVs on retrieved QVs, the approach mitigates issues of incoherence and hallucination common in term expansion-based QV generation. Experiments on TREC DL'19 and DL'20 demonstrate that RAQG-QPP significantly outperforms existing QV-based prediction approaches, achieving up to a 30% improvement on neural ranking models like MonoT5.
Forget term expansion: leveraging retrieved queries and LLMs to generate query variants boosts Query Performance Prediction by up to 30% on neural rankers.
Query Performance Prediction (QPP) estimates the retrieval quality of ranking models without the use of any human-assessed relevance judgements, and finds applications in query-specific selective decision making to improve overall retrieval effectiveness. Although unsupervised QPP approaches are effective for lexical retrieval models, they usually perform weaker for neural rankers. Recent work shows that leveraging query variants (QVs), i.e., queries with potentially similar information needs to a given query, can enhance unsupervised QPP accuracy. However, existing QV-based prediction methods rely on query variants generated by term expansion of the input query, which is likely to yield incoherent, hallucinatory and off-topic QVs. In this paper, we propose to make use of queries retrieved from a log of past queries as QVs to be subsequently used for QPP. In addition to directly applying retrieved QVs in QPP, we further propose to leverage large language models (LLMs) to generate QVs conditioned on the retrieved QVs, thus mitigating the limitation of relying only on existing queries in a log. Experiments on TREC DL'19 and DL'20 show that QPP enhanced with RAQG outperform the best-performing existing QV-based prediction approach by as much as 30% on neural ranking models such as MonoT5.
