Red-Teaming & Adversarial Robustness - Weekly Roundup

The Gaussian kernel's smoothness can lead to catastrophic overconfidence in predictive uncertainty, making it a risky default choice for regression tasks.

SecureDrive-FL withstands Man-in-the-Middle attacks without sacrificing accuracy, achieving a remarkable balance of privacy and performance in federated learning.

Daydreaming can reconstruct agent skills with 86.8% accuracy using only the final outputs, revealing critical vulnerabilities in how skills are protected.

Adversarial training can be achieved without input gradients, cutting computational costs dramatically while maintaining model robustness.

RedEvoAgent evolves red-teaming skills in real-time, outpacing traditional methods and enhancing safety against sophisticated LLM threats.

The Gaussian kernel's smoothness can lead to catastrophic overconfidence in predictive uncertainty, making it a risky default choice for regression tasks.

SecureDrive-FL withstands Man-in-the-Middle attacks without sacrificing accuracy, achieving a remarkable balance of privacy and performance in federated learning.

Daydreaming can reconstruct agent skills with 86.8% accuracy using only the final outputs, revealing critical vulnerabilities in how skills are protected.

Adversarial training can be achieved without input gradients, cutting computational costs dramatically while maintaining model robustness.

RedEvoAgent evolves red-teaming skills in real-time, outpacing traditional methods and enhancing safety against sophisticated LLM threats.

ModelAudit outperforms its competitors by delivering 100% definitive security decisions, revealing significant gaps in the capabilities of existing scanners.

Over 31% of LLM-driven attempts to manipulate PLCs result in sustained physical impacts, exposing alarming vulnerabilities in industrial control systems.

JUDGESTEALER reveals that efficient extraction of LLM judging capabilities is possible without extensive querying, achieving remarkable accuracy across multiple evaluation protocols.

Merging aligned models can inadvertently expose them to jailbreak attacks rooted in their shared pretrained foundations, with a new method that exploits this vulnerability showing high transfer success rates.

A unified framework that links adversarial progression to mission-risk prioritization, enhancing decision-making in cyber defense scenarios.

Metamorphic viruses can now evolve into infinitely diverse forms, rendering traditional antivirus solutions nearly obsolete.

Elevating low-level malicious instructions can completely compromise agent behavior, achieving all attack objectives across multiple harnesses.

Attackers can exploit low-ASR backdoors to bypass state-of-the-art defenses, revealing a critical flaw in current security paradigms.

SPA's innovative plan-first approach eliminates nearly all attack vectors for persistent LLM agents, revealing a critical breakthrough in securing AI interactions with untrusted data.

Indirect prompt injections can exploit framing strategies to bypass defenses, turning a 0% success rate into 100% with simple rewording techniques.

Shared rank-one weight reuse in TEE-offloaded LLMs can be exploited to reconstruct protected parameters with minimal accuracy loss, challenging existing obfuscation strategies.

Achieving an F1-score of 89.76% with a Random Forest model reveals the potential of hybrid anomaly detection in military cyber operations.

FIDA's innovative approach allows backdoor attacks to evade traditional defenses while preserving the functionality of facial recognition systems.

Malicious semantics can be stealthily injected into videos over time, revealing a critical vulnerability in I2V generation models that traditional single-frame attacks overlook.

Attackers can now induce specific failure behaviors in VLA models with unprecedented precision, revealing a new dimension of vulnerability in AI systems.

Achieving a 2.37% boost in Union accuracy over the best baseline, Robust CurveMoE reveals a new paradigm for balancing adversarial defenses across multiple norm constraints.

Stealthy attacks can compromise predictive models, but a new switched model shows improved resilience by explicitly modeling attack probabilities.

Forget-Retain Alignment Gap reveals that the structure of weight updates, not just their distance, is key to preventing LLMs from relearning forgotten information.

LLMs can now proactively correct user input errors, significantly boosting their accuracy and reliability in real-world applications.

Flipping just a few bits can inflate LLM output by over 5900%, revealing a critical vulnerability in Mixture-of-Experts architectures.

Adversarial attacks can manipulate code retrieval systems, reducing their effectiveness by up to 77% without altering code functionality.

SkillShield reduces malware-generation severity from 3.37 to 0.58 while achieving a mere 0.14% safety-refusal rate across 731 benign tasks.

Backdoor detection in self-supervised encoders can be achieved with remarkable accuracy by leveraging generative priors, reducing the need for prior knowledge about the encoder or attack strategy.

Extracting LLM assets from edge AI chips is not just possible; it's alarmingly straightforward with laser voltage imaging, exposing a significant security vulnerability in current AI deployments.

A staggering 77% of confident classifier decisions are susceptible to manipulation by a single token, revealing critical vulnerabilities in prompt injection detection.

Explicitly characterizing adversaries could transform how we evaluate risks associated with releasing open-weight AI models.

Self-evolving coding agents can unwittingly amplify malicious skills, leading to a self-propagating worm effect that evades traditional defenses.

Diverse refusal prefixes can significantly enhance the stability of AI models' refusal behaviors, making them more resistant to adversarial attacks.

Existing detectors fail to reliably distinguish between integrity attacks and sensor faults, with some methods performing at chance level on the most challenging cases.

Trace analysis reveals that up to 87% of flag recoveries in CTF challenges are achieved through shortcuts, not genuine exploitation.

Prompt framing is the leading factor in jailbreak vulnerabilities, revealing critical gaps in multimodal safety alignment that could have serious implications for real-world applications.

A stealthy backdoor attack exploits the batch-dependent behavior of Vision MoE architectures, achieving high activation success rates while evading conventional defenses.

RTLGuard slashes the Attack Success Rate of poisoned RTL code generation models while ensuring the integrity of the generated hardware remains intact.

Attack success rates drop significantly as this framework evolves its defenses in real-time, adapting to new jailbreak strategies without retraining.

Verdict-only evaluations of automated code reviewers can significantly misrepresent their security effectiveness, with PRGuard revealing 1.38 times more target vulnerabilities than existing tools.

Leveraging internal safety neurons, NeuronFuzz achieves up to a 100% jailbreak discovery rate, revolutionizing LLM safety evaluation efficiency.

LLM-enhanced GNNs may boost performance but also expose sensitive information, revealing a stark vulnerability to privacy attacks that traditional models avoid.

ReDiR slashes attack success rates to under 8% by embedding trajectory-level safety insights directly into the action generation process.

LMSM reduces attack success rates by over 90% while maintaining nearly full throughput, revolutionizing how we secure LLM deployments.

A small gyroscope perturbation can lead to meter-scale displacement in UAVs, revealing a critical vulnerability in flight control systems.

Phantom Navigator achieves covert UAV redirection with unprecedented precision, outpacing traditional methods that often fail in real-world applications.

Trust-aware monitoring can significantly enhance multi-robot routing resilience against localization spoofing, restoring expected performance even when adversaries are present.

Subgraph construction, not GNN reasoning, is the Achilles' heel of KGQA systems, accounting for over 99% of performance collapse under adversarial conditions.

Adversarial robustness evaluations in finance can vary by over 700 times depending on the evaluation protocol used.

Supervised UQ ensembles can drastically improve LLM hallucination detection, achieving superior performance with as few as 100 labeled instances.

A staggering 72.9% of medical chain-of-thought rationales fail to influence model answers, raising critical questions about their role in clinical reasoning.

Backdoor attacks in MLLMs can be effectively neutralized without prior knowledge of the trigger, achieving near-perfect defense in most scenarios.

Security-oriented prompts may reduce invalid outputs but paradoxically increase the prevalence of low-severity vulnerabilities in LLM-generated code.

Quantum algorithms are essential for identifying high-correlation cryptanalytic approximations, as classical methods face exponential query limitations.

Sparse backdoor attacks can misclassify over half of triggered anomalies while keeping clean accuracy losses below 2%, revealing a stealthy threat in federated learning systems.

A dual-layer architecture that completely eliminates revocation attacks and blocks all prompt-injection attempts, ensuring safer interactions for LLM agents in web environments.

A unified model that adapts to the interplay between perceptual fidelity and prompt alignment can achieve state-of-the-art performance while revealing interpretable insights into human judgment.

Certifiable randomness can now be achieved unconditionally against low-query-depth quantum adversaries, eliminating reliance on unproven conjectures.

StepGuard slashes attack success rates by over 77% while maintaining nearly all utility, setting a new standard for safety in LLM-based agents.

Cloned outputs from LLMs can be detected with over 87% accuracy using behavioral signatures, revealing significant vulnerabilities in prompt security.

Transferable adversarial examples can exploit vulnerabilities in federated learning, but a new defense mechanism shows how to turn this threat into a strength.

GNN-based NIDS can be significantly hardened against structural adversarial attacks through a novel adversarial training approach that mimics real-world attack scenarios.

Attnlocate reveals that LLMs can effectively guard against injection attacks by pinpointing malicious behavior-guiding instructions in real-time, achieving impressive detection metrics across diverse models.

No trading system architecture is inherently safe; adversarial signals can compromise decisions across all roles, revealing a critical vulnerability in multi-agent setups.

RAGSentinel can filter out adversarial documents with high precision, ensuring that retrieval-augmented generation systems remain robust against sophisticated attacks.

NeuronGuard slashes attack success rates to near-zero while maintaining task performance, revolutionizing LLM safety alignment.

Watermarking can expose hidden vulnerabilities in audio deepfake detection systems, leading to significant performance drops that vary by dataset.

RAG systems are not just enhanced by external knowledge; they are also vulnerable to targeted attacks that can compromise accuracy, privacy, and fairness.

Even the strongest Android GUI agents falter under dynamic runtime anomalies, revealing critical vulnerabilities in their design.

Misanthrope outperforms traditional feature detectors by preventing the identification of people, thus safeguarding privacy while maintaining high image matching accuracy.

Psychological jailbreaks reveal a new frontier in LLM vulnerabilities, achieving an 87.3% success rate through multi-turn persuasion tactics.

Recovering up to 80 bits of the AES128 key under KPA is now feasible in practical timeframes, challenging assumptions about AES's security.

Achieving 99.86% accuracy with a 0.136% false-accept rate, EGAMA-RC redefines malware triage by prioritizing risk and novelty over mere classification performance.

Adversarial prompts can exploit Gumbel-based verification, nearly doubling information leakage and undermining static defense thresholds.

LLMs are more likely to comply with unethical requests when benign framing tokens overshadow cue-tokens, revealing a hidden vulnerability in their alignment.

NICWhisper reveals that electromagnetic emissions from network interface cards can effectively identify network threats, achieving over 80% accuracy without analyzing packet data.

Choosing the right codec for face image compression can mean the difference between a 6.9% and a 98% false non-match rate, depending on your byte budget.

AutoSaddler achieves up to 10% performance improvement in LLM agents by automatically optimizing harnesses based on execution failure signals.

Semantic Overlays can reduce prompt injection attack success rates from 34.8% to just 6.6%, all while keeping the model's output readable and accurate.

Slopsquatting risks are significantly mitigated with a two-layer detection system that achieves 76% hallucination-free code generation, even in adversarial contexts.

Even top-performing models can produce misleading explanations, revealing a hidden crisis in the trustworthiness of Explainable AI.

Stitch achieves a remarkable 31% boost in pivot attack detection accuracy while keeping false positives to an astonishing 0.006%.

Only 4-16% of security rules in CLAUDE.md align with built-in controls, exposing a critical gap in rule enforceability.

Tighter bounds on instance encoding invertibility reveal that deterministic encoders can be just as secure as their randomized counterparts, transforming our understanding of data privacy techniques.

TrustShift attacks can manipulate agent trust dynamics, achieving a staggering 69.5% success rate before being mitigated by a novel defense framework.

ROBBIN achieves nearly 90% attack success while preserving over 83% accuracy, making it a game-changer for reliable backdoor attacks across varying DRAM devices.

Model robustness against training-time data contamination varies dramatically, challenging the assumption that clean data performance predicts real-world reliability.

A single interaction is all it takes for an adversary to manipulate LLM memory systems, steering outputs toward a pre-specified target with alarming precision.

Some protocols are vulnerable to traditional Dolev–Yao attacks but remain secure against rational attackers, revealing a critical gap in conventional security assessments.

TEE-X achieves GPU-level inference latency for large vision models while ensuring robust security in edge applications, transforming the landscape of model deployment.

As the number of sampled outputs increases, the risk of selecting unsafe outputs becomes asymptotically certain, even with seemingly effective safety proxies in place.

Task-asymmetric miscalibration in on-device language models leaves developers blind to when their outputs can be trusted, with confident-wrong outputs indistinguishable from confident-correct ones.

DFL-C not only ensures global model consistency in decentralized federated learning but also outperforms existing solutions in the face of Byzantine attacks.

RAG-based defenses can significantly reduce package hallucination rates in LLM-generated code, but only if matched to the specific threat model and utility requirements.

AdaptPrint reveals hidden LLM identities with up to 92.1% accuracy, transforming the landscape of security assessment for black-box AI services.

AEGIS can effectively defend against Indirect Prompt Injection attacks while maintaining low latency and high utility, a breakthrough for LLM safety.