Berkeley AI Research (BAIR)

State-of-the-art Vision-Language Models fall short in real-world robotic applications, revealing critical gaps in their reasoning capabilities.

Synthesizing 48,000 interaction trajectories without human input enables a humanoid robot to learn complex loco-manipulation tasks effectively.

Grounded verification in TEXEDO enables humanoid robots to execute complex motions that are both semantically aligned with text prompts and physically feasible.

Libretto transforms symbolic music generation into a structured, editable process, enabling LLMs to create and revise music with unprecedented precision and control.

Human videos can now be transformed into actionable manipulation data for robots, overcoming traditional barriers in hand-object interaction estimation.

Spatial attention in VLMs is nearly irrelevant to accuracy, with self-consistency emerging as the true indicator of reliability.

VisualClaw slashes API costs by 98% while boosting accuracy, transforming how VLMs can operate in real-time environments.

Extracting action signals from 32,041 hours of human video enables CAIP to outperform leading vision encoders in robotic manipulation tasks by over 30%.

Tactile-reactive policies can boost robotic manipulation success rates by over 30% through innovative data collection and a new Mixture-of-Transformers architecture.

FTP-1 not only excels on familiar tactile sensors but also achieves unprecedented success on unseen setups, redefining the potential for cross-sensor generalization in robotic manipulation.

Surflo revolutionizes 3D surface reconstruction by enabling arbitrary-resolution outputs from a single global state, outperforming traditional methods in both speed and accuracy.

StreamForce achieves real-time video generation with physical control, outperforming traditional models in both responsiveness and realism.

Stateful Visual Encoders enable VLMs to leverage prior visual context, leading to substantial performance gains in multi-image tasks.

Forget task-specific fine-tuning – teaching VLMs basic geometry yields a +29% boost on spatial reasoning benchmarks.

Camera pose, largely ignored in video LLMs, unlocks significant gains in spatial reasoning and even improves general video QA when used as a lightweight supervisory signal.

Adversarial clothing with non-overlapping visible-thermal patterns can reliably evade RGB-T detectors, even transferring across different fusion architectures.

The dream of universal representations across modalities may be just that: scaling up datasets and relaxing constraints reveals that models trained on different modalities learn rich, but fundamentally different, representations of the world.

Unlock 36% better video depth estimation and 20% better camera pose estimation by simply letting your model learn from its own unlabeled video predictions.

Forget hyperparameter tuning – autonomous research reveals that bug fixes and architectural tweaks unlock far greater gains in multimodal agent memory.

LVLMs can be made significantly less prone to hallucinations, without any training, by explicitly grounding them in visual evidence and iteratively self-refining their answers based on verified information.

Forget simulated manipulation—ManipulationNet offers a global infrastructure for benchmarking robots in the real world, complete with standardized hardware and software, to finally measure progress toward general manipulation.

Multimodal web agents are surprisingly vulnerable to cross-modal attacks, but a novel adversarial training approach can double task completion efficiency while mitigating these risks.

An educational RAG system achieves 84% accuracy in answering student questions with minimal human editing, suggesting a practical path towards scalable AI-assisted teaching.