Yazan Boshmaf

SSP-Bench reveals systematic failures of static evaluation, including near-zero correlation in safety rankings due to construct mixing, strong safety--over-refusal coupling, and hidden within-family regressions, and shows that static benchmarks can misrepresent model behavior, motivating dynamic, deployment-relevant evaluation.

A single pipeline configuration choice can swing a model's benchmark score by more than 80 percentage points, revealing that most cybersecurity LLM rankings reflect harness implementation quirks rather than actual capability.

Code LLMs can be reliably poisoned to generate vulnerable code by subtly manipulating the style of the input prompt, even while maintaining strong performance on standard benchmarks.