Search papers, labs, and topics across Lattice.
This paper introduces KeyPooling, a measurement method that investigates how customer identities are managed within LLM API relays, revealing that shared provider credentials lead to significant cache state exposure among relay customers. The study tested five open-source gateways connected to major providers and found that none isolated customers by default, resulting in cross-customer cache reads across multiple identities. The findings highlight critical vulnerabilities in API relay architectures, suggesting that without strict domain enforcement, customer data can be compromised through shared caches.
KeyPooling uncovers that shared credentials in LLM API relays can expose customer cache states, revealing a systemic vulnerability that threatens data privacy.
Large language model (LLM) API relays authenticate customers separately but often forward requests through shared provider credentials. Providers scope prompt caches to upstream principals and namespaces, so relay customers mapped to one cache identity can observe each other's cache state. Prior work showed cache sharing at selected endpoints but did not identify which credential, pool, adapter, or nested hop controls the finalidentity. We present KeyPooling, a measurement method that traces customer identity through cache lookup and write, verifies runtime transformations, and tests one predicted identity component at a time. Across five open-source gateways connected to OpenAI and Anthropic, none bound customers to upstream credentials by default; under a shared credential, all five exposed cross-customer cache reads for both providers. Principal and namespace splits, pool associations, and adapter and nested-relay contrasts localized the controlling transformations. In an outcome-independent weekly OpenRouter frame, tests covered 80.5% of eligible token volume and found cross-account reads for 12 of 28 labels carrying 33.7% of volume. On one production route, a controlled procedure recovered eight consecutive target positions without target access. Broader tests identify cache granularity, routing, rate limits, attribution, and budget as conditions for token-by-token recovery, not security controls. We derive a defense contract: every customer must enter a provider-enforced domain, or a namespace derived from authenticated identity must survive every final cache lookup and write. Placing this split after reusable public prefixes preserved most modeled reuse at a 1.7-2.5% cost increase.
