Berkeley AI Research (BAIR)

Substantial challenges in task generalization and visuomotor robustness are revealed, highlighting critical gaps in current dexterous manipulation benchmarks.

Continuous scoring from LLM-as-a-Verifier leads to state-of-the-art verification accuracy and improved sample efficiency in reinforcement learning tasks.

RoboDojo reveals that existing benchmarks fail to capture the full spectrum of robot manipulation capabilities, paving the way for more robust evaluations that bridge the gap between simulation and real-world performance.

Claw-like agents are vulnerable to severe security breaches, with malicious plugins achieving a 100% success rate in attacks.

Retraining generative models with different seeds can shift FID scores dramatically more than merely resampling, revealing a hidden layer of randomness in model evaluation.

Kappa deflation reveals that LLM-as-a-Judge models may be overstating their discriminative abilities by up to 41 percentage points.

Spatial attention in VLMs is nearly irrelevant to accuracy, with self-consistency emerging as the true indicator of reliability.

Current AI agents excel in structured tasks but falter at generating novel insights and tackling open-ended scientific challenges.

LLMs reveal surprising strengths and weaknesses in analyzing security logs, with performance heavily influenced by model design choices.

LLMs get *worse* at forecasting high-stakes events like epidemics and financial crises as they get more capable, because they aggressively extrapolate growth and overestimate tail risk.

LLMs subtly manipulate political discourse by framing arguments asymmetrically, but this can be significantly reduced with a novel consistency training approach.

LLM-powered query reformulation, a hot topic in IR, often fails to translate gains from lexical to neural retrieval, and bigger models don't always help.

LLMs exhibit Pareto-like tradeoffs in medical diagnosis, where neutralizing user prompts to improve plausibility and conciseness can simultaneously reduce coverage of critical conditions.

The dream of universal representations across modalities may be just that: scaling up datasets and relaxing constraints reveals that models trained on different modalities learn rich, but fundamentally different, representations of the world.

Current LLM detection methods in peer review are fooled by hybrid human-AI workflows, mistaking AI-written text for AI-originated ideas.

Agentic data science pipelines often reach falsely optimistic conclusions, but two simple sanity checks can expose these unsupported claims by testing if the agent can reliably distinguish signal from noise.

AI audit standards can fail to ensure responsible AI practices due to vague requirements and undefined terms, even while appearing compliant.

Poisoning a personal AI agent's Capability, Identity, or Knowledge triples its vulnerability to real-world attacks, even in the most robust models.

LVLMs can be made significantly less prone to hallucinations, without any training, by explicitly grounding them in visual evidence and iteratively self-refining their answers based on verified information.

Existing QA benchmarks are too easy for LLMs, so iAgentBench offers a more realistic challenge by requiring agents to synthesize information from multiple sources on high-traffic topics.

Forget simulated manipulation—ManipulationNet offers a global infrastructure for benchmarking robots in the real world, complete with standardized hardware and software, to finally measure progress toward general manipulation.

Forget temperature scaling: JUCAL calibrates aleatoric and epistemic uncertainty in classifier ensembles, achieving SOTA results with significantly smaller ensembles and lower inference costs.

LLMs evaluating job candidates exhibit significant bias against hedging language, docking candidates by 25.6% on average, even when the content is equivalent.