Search papers, labs, and topics across Lattice.
This paper benchmarks various categorical filtered approximate nearest neighbor (ANN) methods across multiple datasets and predicates, revealing that no single method consistently outperforms others. To address this variability, the authors introduce a query-aware routing framework that utilizes a lightweight machine learning model to predict the recall of candidate methods based on the query context. The proposed router achieves state-of-the-art recall and query per second (QPS) balance across five unseen validation datasets, with minimal latency overhead, enhancing the efficiency of vector databases and retrieval-augmented generation systems.
A query-aware routing framework can significantly optimize filtered ANN searches, achieving superior recall and QPS balance without added latency.
Filtered ANN search, which combines vector similarity with attribute predicates, is a core primitive in modern vector databases and retrieval-augmented generation. We benchmark all major categorical filtered ANN methods across multiple datasets under three predicates and find that no single method dominates. Moreover, even within a single dataset and predicate type, the best method for a query can vary. Therefore, we propose a query-aware routing framework. A lightweight ML model predicts each candidate method's recall on the query, and the router consults an offline benchmark table that maps every method and parameter setting to its measured recall and QPS, then selects the method with the best recall--QPS trade-off. Our ablation study narrows 22 candidate features to a minimal set of three and we adopt regression rather than classification as the prediction target to sharpen accuracy. Our model is trained on six real-world datasets and applied to five unseen validation datasets. The final result shows that our router achieves state-of-the-art recall and QPS balance across all five validation datasets compared to existing filtered ANN baselines, while incurring negligible latency overhead.
