Search papers, labs, and topics across Lattice.
This paper addresses the vulnerabilities of black-box LLMs to distillation attacks, where attackers can train a student model using outputs from a teacher model accessed via API. The authors critique existing output perturbation defenses for their lack of a unified threat model, which complicates their evaluation and robustness against realistic adversaries. By proposing a comprehensive threat model framework that categorizes attackers based on query budget, data budget, and interface profile, they demonstrate that the effectiveness of antidistillation defenses is contingent on the threat model employed, urging for more precise specifications in future research and policy frameworks.
The effectiveness of antidistillation defenses hinges on the threat model used, revealing that current defenses may offer a false sense of security.
Black-box LLMs (accessible only via API) are vulnerable to distillation attacks, in which an attacker queries the model and trains a student on its outputs. A recent line of work proposes output perturbation defenses that modify the teacher's output to reduce student performance while preserving utility for legitimate users. As a relatively new family of approaches, output perturbation defenses lack a shared threat model, making it difficult to compare them, reason about composing them with other attacks, or evaluate their robustness against realistic adversaries. This underspecification matters beyond technical evaluation: when defenses are deployed to protect intellectual property or justify regulatory compliance, an imprecise threat model can create a false sense of security. We propose a threat model framework that describes attackers along three dimensions: a query budget, a data budget, and an interface profile that captures how attackers interact with the API. Using antidistillation sampling as a case study, we show that whether the defense is considered effective depends on the assumed threat model. We argue that future work on distillation defenses, along with any governance or policy frameworks built around them, should explicitly specify and stress-test attacker capabilities along our three dimensions.
ETH