NVIDIA Research

This work introduces PosteriorBench, a benchmark for evaluating the distributional accuracy of generative inverse solvers, and reveals substantial distribution-matching gaps across current solvers, while showing that neural operators improve resolution robustness, and guidance weights and generation noise are key to posterior-variance calibration.

By harmonizing diverse video datasets and architectures, SolarWM enables real-time interactions in video world models, pushing the boundaries of long-horizon inference.

Achieving a staggering 90.4% reduction in robot-motion error, Hydra-0 redefines how we model and control robotic actions across varied environments.

FedCoRe can recover nearly half of the performance lost due to missing ECG or CXR data in federated healthcare models, transforming how we handle incomplete patient information.

UniProbe slashes object hallucinations in LVLMs by 55% during generation, all while operating with minimal latency.

WorldTrace redefines memory management in video world models, achieving up to 19.5% better episodic recall without any retraining.

VLMs struggle to provide accurate annotations in video games, revealing significant gaps in their understanding of dynamic environments.

Annotating video game datasets with VLMs can transform the way RL agents learn by simplifying reward extraction and conditioning.

GROVE achieves state-of-the-art performance in memory retrieval for wearable assistants by seamlessly integrating reactive and proactive memory access from continuous video streams.

Achieving state-of-the-art video generation with significantly improved diversity and speed, PDD redefines the efficiency of diffusion models.

Jointly optimizing visual token selection and LLM computation can drastically reduce inference costs while enhancing performance in multimodal tasks.

AV-Flamingo outperforms existing models on complex audio-visual tasks, revealing that size isn't everything when it comes to reasoning capabilities.

AeroAct achieves real-time language-conditioned quadrotor navigation by predicting flight actions directly from visual and language inputs, without the need for future video generation.

Adaptive routing in transformer-based models can drastically enhance the efficiency of PNI prediction while maintaining high accuracy in capturing subtle imaging features.

ARDY achieves real-time, controllable 3D human motion generation that outperforms existing methods in both fidelity and flexibility.

Flowley not only streamlines video-to-audio generation with a single-stage architecture but also sets a new benchmark for audio quality by leveraging sound-aware captions.

Current avatar systems are more diverse than ever, yet foundational prior learning is often overlooked in discussions of photorealistic digital humans.

Current VLMs struggle with specialized domains, failing to adapt effectively in both zero-shot and ICL scenarios, revealing critical gaps in their spatio-temporal reasoning abilities.

VLM-AR3L outperforms traditional reward learning methods by integrating absolute and relative rewards, enabling agents to navigate complex tasks with greater efficacy.

Dynamic token editing in image synthesis could redefine how we approach high-resolution generative models.

Tri-serve redefines energy efficiency in multimodal inference by addressing hidden power inefficiencies, achieving a 22% boost without latency trade-offs.

Policies trained in SimFoundry's automated environments achieve up to 40% higher success rates in real-world tasks by leveraging affordance-preserving scene variations.

NeuMatEx outperforms PBR techniques by extracting complex neural materials with unprecedented visual fidelity and precision from multi-view images.

Current multimodal conversational models miss critical emotional cues, revealing a significant gap in their ability to engage in nuanced human-like dialogue.

Self-correcting models can achieve unprecedented fidelity and plausibility in generative tasks by actively learning from their own alignment errors.

SR-REAL's dual-path reasoning framework allows spatial VLMs to excel in both linguistic deduction and 3D geometric inference, significantly enhancing performance on complex spatial reasoning tasks.

Extracting action signals from 32,041 hours of human video enables CAIP to outperform leading vision encoders in robotic manipulation tasks by over 30%.

Tactile-reactive policies can boost robotic manipulation success rates by over 30% through innovative data collection and a new Mixture-of-Transformers architecture.

SPARC reduces noisy labels by leveraging task structure, enabling robots to learn from more reliable demonstrations and outperforming traditional methods in real-world applications.

Decoupling modality processing in VLA models leads to a staggering 95.2% success rate in complex manipulation tasks, far surpassing traditional synchronous approaches.

VLMs trained on the new 4DP-QA dataset show marked improvements in understanding complex 4D scenes, revealing the critical role of disentangling motion dynamics.

GRAIL achieves an impressive 84% success rate in real-world object pick-up tasks using only synthetic data, revolutionizing humanoid robot training.

Cosmos 3 sets a new benchmark for omnimodal models, outperforming existing state-of-the-art in both Text-to-Image and Image-to-Video tasks.

Current vision-speech agents are surprisingly bad at mimicking the subtle, real-time audio-visual cues that make human conversation feel natural.

Grounding boosts spatial reasoning in VLMs: explicitly linking language to 2D and 3D scene elements lets models decompose complex spatial problems and improve performance even on non-grounded tasks.

Masking just 5% of attention heads in vision-language models tanks performance on long-context tasks, revealing a surprisingly sparse and critical set of "multimodal retrieval heads" that attend to both text and images.

Ditch slow, token-by-token box generation: LocateAnything's Parallel Box Decoding (PBD) boosts VLM grounding speed and accuracy by decoding entire bounding boxes at once.

Ditch the clunky tool-use pipelines: STORM teaches video-language models to reason about space and time using *internalized* latent trajectories, slashing inference costs while boosting accuracy.

A generative model of human physiology not only beats existing clinical risk scores at predicting disease, but also accurately simulates the effects of clinical interventions, paving the way for personalized medicine.

VLN agents can navigate more accurately in zero-shot settings by "looking forward, now, and backward," mimicking human navigational strategies.

Multimodal models can now achieve state-of-the-art performance in real-world tasks like document understanding and audio-video comprehension with significantly reduced inference latency thanks to novel token-reduction techniques.

Audio-language models can now reason about 30-minute-long audio clips with timestamp-grounded intermediate steps, unlocking a new level of fine-grained understanding.

Finally, a method disentangles dynamic egocentric scenes into background, hand, and object components, enabling fine-grained understanding and editing.

Swap out slow, one-token-at-a-time generation in VLMs for a 6x speed boost, without sacrificing quality, using a surprisingly simple direct conversion to block-diffusion decoding.

Finally, a video generation model lets you puppeteer objects and their reactions independently, all while freely moving the camera.

Forget simulated manipulation—ManipulationNet offers a global infrastructure for benchmarking robots in the real world, complete with standardized hardware and software, to finally measure progress toward general manipulation.

Forget monolithic LoRAs: LoRWeB dynamically mixes a basis set of LoRAs to unlock SOTA generalization in visual analogy tasks.

Forget synthetic data that looks like it came from a PS2 game: NVIDIA's new Cosmos-Predict2.5 generates high-fidelity videos for training embodied AI, opening the door to more realistic and reliable simulations.