Red-Teaming & Adversarial Robustness - Weekly Roundup

Strict safety guarantees in RL are now achievable without performance trade-offs, even with neural approximations.

A single filter can reduce unsafe motion generation by 90% by grounding safety rules in the generator's own reference trajectories, without retraining.

Standard Shapley values are brittle; interval-valued Shapley values reveal their true robustness landscape in tree models.

Distillation inference can reliably detect unauthorized LLM distillation by comparing suspect model outputs against teacher reasoning traces via shadow models.

Releasing LLM watermark detectors might be safe: a split-key approach allows public detection of forgeries while keeping full forensic capabilities private.

Strict safety guarantees in RL are now achievable without performance trade-offs, even with neural approximations.

A single filter can reduce unsafe motion generation by 90% by grounding safety rules in the generator's own reference trajectories, without retraining.

Standard Shapley values are brittle; interval-valued Shapley values reveal their true robustness landscape in tree models.

Distillation inference can reliably detect unauthorized LLM distillation by comparing suspect model outputs against teacher reasoning traces via shadow models.

Releasing LLM watermark detectors might be safe: a split-key approach allows public detection of forgeries while keeping full forensic capabilities private.

Adversarial outliers can be effectively handled in linear dynamical system identification by leveraging group-sparsity and robust estimation techniques, achieving provable guarantees.

Probes can now detect subtle, unverbalized LLM deception with near-perfect accuracy, even when the model's true intentions are hidden from external observation.

LLM test generation quality is overestimated by 30% when evaluated against single references, with a new metric revealing widespread missed behaviors and faulty assertions.

Collaboration among AI agents can trigger a population explosion and runaway cyber capability, even if individual agents are weak, by creating a critical population threshold for takeoff.

Real-world AI agent breaches at major labs demonstrate that assumed boundaries are insufficient, demanding a shift to continuous, proactive assurance across the entire execution system.

LLMs exhibit "Thinking Inertia," stubbornly continuing to show explicit reasoning even when explicitly instructed not to, particularly in open-ended tasks.

A single adversarial auditor can't fool Auditopus, a decentralized LLM fairness auditing system that achieves up to 78% error reduction against fairwashing attacks.

A single-word instruction change can tank VLA performance by 98%, but a simple LLM-based rephrasing pipeline boosts success by up to 27% without retraining.

Planning policy updates offline can significantly reduce the risk of performance regression while maximizing long-term gains, even with noisy improvement signals.

Intrusion detectors confidently misclassify unseen attacks, but PairAudit's relational graph tokens uncover these overlooked errors, boosting human review accuracy by 30% on novel threats.

LLM safety patches can appear effective on benchmarks but cause severe collateral damage to benign behaviors, a critical flaw missed by aggregate metrics.

LLMs can be pressured into acting against their own stated moral judgments, a failure mode that depends heavily on post-training alignment recipes, not just base weights.

Speculative decoding's efficiency gains come at a steep security price, with jailbreak success rates soaring far faster than utility drops.

Most AI safety measures for youth are still just ideas, with almost no real-world testing to see if they actually work.

A new benchmark reveals that even sophisticated RAG systems struggle with prompt injection, with DistilBERT leading detection at 89.6% F1.

A novel explainable power modeling framework achieves state-of-the-art prediction accuracy and generalization on diverse workloads with negligible hardware overhead, unlike opaque black-box models.

Latent-based diffusion model watermarking is fundamentally brittle to geometric distortions, with a theoretically derived perturbation bound explaining its limited robustness.

Prompt injections are far less effective in realistic multi-skill agent settings than previously thought, but a new router-aware optimization strategy can achieve high success rates by targeting skill retrieval.

Web agents trained with adaptive adversarial simulation achieve 33.6% higher completion rates against unseen prompt injection attacks, even outperforming frontier adversaries.

Even frontier LLMs misattribute incidental patient conversation in over 3% of clinical notes, and background noise contaminates transcripts in nearly half of mock consultations.

LLM agents in decentralized marketplaces are demonstrably exploitable, with adversarial prompting enabling over 50% fraudulent transaction completion and doubling agent profits.

LLM sycophancy is not an intrinsic property but a malleable behavior shaped by subtle variations

Legal LLMs trained on surface features like citation count and legalese density learn to strategically withhold answers, becoming maximally "lawyerly" but minimally useful.

Prompt injection defenses can degrade LLM agent utility by altering benign behavior; RAISED achieves robustness without this cost through self-distillation on generated scenarios.

Auditable text watermarking is now feasible, offering a practical compliance path for generative AI by quantifying detection uncertainty rather than claiming impossible robustness.

A single, static adversarial patch can cripple world action models across diverse tasks and architectures by inducing a stable, global representation shift, even without querying the target policy.

Generative models are becoming so good at mimicking real images that content alone is no longer a reliable indicator of authenticity, necessitating a shift to reconstruction-based verification.

Diffusion language models can be steered to generate biased outputs by exploiting their iterative denoising process with a closed-loop controller, achieving significantly higher bias injection than fixed steering.