Search papers, labs, and topics across Lattice.
This paper investigates the challenge of identifying the source of text generated by various black-box LLMs using a novel framework called READER, which leverages a frozen proxy LLM to decode authorship evidence from model outputs. By employing Bayesian Evidence Accumulation and temporal filtering of token states, READER achieves significant improvements in provenance accuracy, reaching up to 84.0% top-1 accuracy when aggregating responses from multiple prompts. The findings indicate that stronger LLMs inherently reveal more structured authorship information, paving the way for reliable attribution in dynamic query environments.
READER reveals that even frozen LLMs can expose rich authorship signals, achieving up to 84% accuracy in identifying model sources from black-box outputs.
As agentic applications increasingly route user tasks through official and third-party LLM APIs, provenance becomes an operational question: which model generated a given black-box response? We study Dynamic Black-Box LLM Provenance: identifying the source LLM from generations elicited by query-varying, non-predefined prompts rather than a fixed input set or benchmark suite. This setting is difficult because prompt semantics dominate the text, while model-specific authorship traces are weak and inconsistent at the surface level. We introduce READER (Robust Evidence-based Authorship Decoding via Extracted Representations), a lightweight provenance framework that treats a frozen proxy LLM as a reader of hidden authorship evidence. READER maps black-box outputs into proxy activation space, temporally filters token states within each response, and performs Bayesian Evidence Accumulation by summing single-response log-posterior evidence across independently sampled prompts. This avoids fragile mean-pooling of prompt-specific representations while preserving the query-wise evidence needed for calibrated confidence. On Agent500, a 50-target dataset built from agent-style prompts, READER reaches $31.0$-$42.4\%$ top-1 accuracy from a single response and $70.0$-$84.0\%$ from 50 responses, substantially outperforming sentence-encoder fingerprints. Scaling across nine proxy readers further shows that stronger LLMs expose more linearly decodable authorship structure, suggesting that authorship perception is already present in frozen LLM representations and can be converted into reliable multi-query attribution.
NUS
Tsinghua AI