Red-Teaming & Adversarial Robustness - Weekly Roundup

Safety guard evaluations can be misleadingly inflated by superficial cues, with simple edits to tool names or command structure significantly altering measured performance and agent behavior.

Robust RL policies for adaptive cancer therapy achieve superior performance and generalization by treating patients as hidden contexts, outperforming existing methods even with noisy biomarkers and shifting disease dynamics.

Agent guards designed for English requests exhibit significant safety failures when faced with multilingual inputs, with smaller models and specialized guards showing the most pronounced language-dependent biases.

Transformers trained with ABNAT learn to mimic clean attention maps, achieving up to 85.77% less attention deviation and 11.28% higher accuracy on noisy analog hardware.

A standard CNN technosignature detector misses 91% of genuine signals when they deviate slightly from the training morphology, and is fooled by terrestrial interference as often as it finds real signals.

Safety guard evaluations can be misleadingly inflated by superficial cues, with simple edits to tool names or command structure significantly altering measured performance and agent behavior.

Robust RL policies for adaptive cancer therapy achieve superior performance and generalization by treating patients as hidden contexts, outperforming existing methods even with noisy biomarkers and shifting disease dynamics.

Agent guards designed for English requests exhibit significant safety failures when faced with multilingual inputs, with smaller models and specialized guards showing the most pronounced language-dependent biases.

Transformers trained with ABNAT learn to mimic clean attention maps, achieving up to 85.77% less attention deviation and 11.28% higher accuracy on noisy analog hardware.

A standard CNN technosignature detector misses 91% of genuine signals when they deviate slightly from the training morphology, and is fooled by terrestrial interference as often as it finds real signals.

Continual learning, aiming to improve deepfake detector accuracy over time, systematically intensifies unfairness across demographic groups.

Existing benchmarks fail to accurately measure edge unlearning effectiveness, with current methods often slower than retraining and standard accuracy metrics proving misleading.

Hundreds of adversarial honesty targets are sufficient to teach LLMs to confess their hidden behaviors and resist pressure to lie, outperforming existing methods without sacrificing utility.

A single adversarial token can bypass most safety filters.

LLM agents can be made robust to complex, multi-turn adversarial attacks on tool use through a co-evolutionary game, outperforming prior methods on safety and capability benchmarks.

LLM agents falter significantly when faced with real-world user emotions, revealing a critical fragility in their long-horizon tool-calling capabilities.

Autonomous driving VLAs can be made conflict-aware by dreaming world models that strictly follow navigation signals, revealing unsafe instruction scenarios.

LLM agents trained on function calling forget nearly 25% of their capability when action names are perturbed, but a simple self-supervised regularization recovers over half of that loss.

Multilingual LLM bias audits are fundamentally flawed: even perfect translations cause stereotype scores to plummet, obscuring true fairness differences.

Language models can be trained to proactively avoid costly errors by learning to "fail gracefully" directly from outcome-level rewards, without needing explicit safety annotations or verifiers.

Adaptive perturbation radius in SAM, guided by local curvature, unlocks superior generalization by escaping sharp minima more effectively without sacrificing convergence speed.

Standard imputation methods fail to predict downstream utility, but task-aware imputation recovers complete-data performance in up to 80% of challenging train-test mismatch scenarios.

Planetary landing controllers trained with adversarial disturbances uncover critical failure modes missed by standard randomization, suggesting a new paradigm for robust aerospace GNC.

A novel geometric analysis reveals how fairness scaling warps client update distributions, enabling structure-aware Byzantine detection beyond simple magnitude filtering in $q$-fair federated learning.

LLMs can learn to manipulate their own internal activations, but this control is limited and often mimicked by external generation.

State-of-the-art coding agents exhibit critical boundary violations, with nearly all misinterpreting dependencies and over half compromising environment integrity or user inputs.

Adversarial image perturbations and decision-based attacks can render state-of-the-art OCR models nearly useless for academic integrity, even against advanced transformer architectures.

LLM agents can be secured against indirect prompt injection by mediating their effects at the system boundary, reducing attack success from 47.0% to 2.7% on AgentDojo without sacrificing utility.

DNN-based IDSs trained in realistic cyber environments exhibit near-perfect in-domain detection but suffer catastrophic performance degradation with minor domain shifts.

A single LLM can now autonomously navigate untrusted computer interfaces and execute sensitive tasks without human oversight, achieving zero attack success rate and higher utility than previous methods.

Even with state-of-the-art defenses, individual records with unique signatures remain highly vulnerable to membership inference attacks in tabular data.

Over-parameterization's impact on robustness is not monotonic; solution geometry and optimization path, not just parameter count, dictate sensitivity in matrix sensing.

Most industrial anomaly detectors collapse under simple brightness or zoom changes, losing up to 34 AUROC points, but a new approach using synthetic anomalies maintains over 92% AUROC.

Future LLM agents can be prevented from reward hacking by using today's models to automatically harden their evaluation environments.