Search papers, labs, and topics across Lattice.
This study reveals that fine-tuning large language models (LLMs) for security classification can inadvertently introduce evasion vulnerabilities that standard evaluation methods fail to detect. By analyzing the Foundation-Sec-8B-Instruct model and its base Llama-3.1-8B-Instruct, the authors demonstrate that fine-tuning leads to the specialization of inherited classification circuits, which, while enhancing performance, creates new attack surfaces sensitive to behavior-preserving transformations. Their findings underscore the importance of monitoring semantic drift during fine-tuning to ensure robust AI security, as models can maintain high accuracy while becoming more vulnerable to specific evasion techniques.
Fine-tuning security classifiers can enhance accuracy but also expand their evasion surface, making them more susceptible to sophisticated attacks.
LLMs fine-tuned for security classification are usually evaluated on held-out examples from the same distribution as their training data. We show that this can miss vulnerabilities introduced by fine-tuning itself: models can learn token-level indicator semantics that preserve canonical accuracy while failing under behavior-preserving transformations such as PowerShell alias substitution, command reconstruction, string construction, execution indirection, and case mutation. We study Foundation-Sec-8B-Instruct and its base model, Llama-3.1-8B-Instruct, on matched PowerShell classification cohorts. Causal interventions localize the classification circuit to a late-attention route inherited from Llama rather than created by fine-tuning. Fine-tuning concentrates and semantically specializes this inherited structure, improving baseline behavior while creating transformation-sensitive attack surfaces. A three-tier evasion benchmark finds Foundation-Sec misses on iwr substitution, Invoke-Expression reconstruction, and case-mutated Invoke-Expression/IEX variants that Llama does not share. We also derive a pre-deployment monitoring method: a linear probe at the classification boundary and an indicator-token sign test identify command families where canonical indicators change role after fine-tuning. These signals prioritize red-team variant generation using only canonical inputs, showing that security fine-tuning can improve task accuracy while expanding the evasion surface. These results caution against treating small task-specific fine-tunes as straightforwardly safer security classifiers: specialization can convert inherited model structure into brittle indicator rules that preserve held-out accuracy while expanding the evasion surface. Robust AI-enabled security will require specifying the full transformation space of the task and monitoring semantic drift through fine-tuning.
