Search papers, labs, and topics across Lattice.
This paper investigates embedding inference attacks (EIA) in information retrieval systems, specifically in a black-box setting where the attacker can only observe the unordered set of retrieved documents. The authors demonstrate that tailored queries can effectively identify the embedding model in use from a predefined set of candidates, even when defensive mechanisms like rerankers are employed. The findings highlight significant vulnerabilities in retrieval-augmented generation systems, revealing that certain queries can bypass the model's defenses against unrecognized inputs.
Tailored queries can expose the embedding model used in retrieval systems, even when adversaries only see unordered document sets.
Embedding models are essential components of modern Information Retrieval (IR) systems, yet they are typically hidden behind APIs. Recent works have shown that dense IR system can lead to security vulnerabilities such as embedding inversion attacks. However, such attacks usually require that the attacker knows the embedding model for the attack to be applicable. In this paper, we study IR systems under a black-box setting in which the adversary observes only the unordered set of retrieved documents, without ranking or similarity scores. We demonstrate that in such contexts, tailored queries allow an adversary to identify which embedding model is in use from a set of known model candidate, which we coin as an embedding inference attack (EIA). We also show that certain queries remain discriminative even when the system includes a reranker as a potential defense mechanism. We further validate our method on a real Retrieval-Augmented Generation (RAG) system, in which the tailored queries bypass the LLM's tendency to reject inputs it does not recognize as well-formed questions. Finally, we propose and evaluate other mitigation strategies such as similarity thresholds.
