CMU Machine Learning

Explicitly grounding the goal object in view unlocks near-expert performance in object-goal navigation, revealing perception rather than policy as the key limitation.

Visual transition reasoning, not just diverse data, unlocks significant multimodal LLM improvements, with just 2,000 examples boosting 22 benchmarks.

Dynamically recomputing embeddings at each denoising step in diffusion transformers halves training compute while achieving state-of-the-art FID scores.

It is shown that the workspace token can be used as a drop-in replacement for observations during deployment, enabling policies to solve memory-intensive tasks without the need for VLM reasoning in-the-loop.

Modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask, suggests modern VLMs already encode much of what is needed for specialized domains; the challenge is learning how to ask.

A simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically, shows that reasoning and retrieval are complementary on rare entities.

Achieving higher novel-view quality in 3D scene generation without requiring dense volumetric representations or ground-truth supervision is a game changer for practical applications.

SuperMap maintains stable object identities in dynamic environments, preventing identity drift and stale semantics in robotic navigation.

Policies may succeed in tasks but still violate deformation tolerances, revealing a critical gap in current evaluation methods for deformable-object manipulation.

Closing the loop on robot manipulation failures, VLCP rewrites control code in real-time, achieving a tenfold increase in success rates over traditional methods.

VLSR's localize-then-reason approach boosts throughput by 9.6X, revolutionizing how we analyze molecular properties from images.

High speech overlap isn't the primary challenge in cocktail-party scenarios; innovative audio-visual strategies and large language models can cut recognition errors by 57%.

H2S achieves a remarkable 48.54 mAP in audio-visual instance segmentation, setting a new benchmark in the field.

VLMs can fabricate highly biased medical diagnoses based solely on demographic descriptors, revealing alarming implications for clinical trustworthiness.

Prediction objectives shape the physical knowledge embedded in latent representations, revealing that some parameters are only captured under specific forecasting conditions.

Proprietary MLLMs may achieve high diagnostic accuracy, but they still struggle with reliable clinical reasoning, revealing significant gaps in their practical utility.

Achieving 19 times lower inference latency than diffusion models, UniGen-AR redefines the efficiency of unified visual generation.

Different views of the same problem reveal hidden reasoning paths, enabling VLMs to achieve unprecedented accuracy in multimodal reasoning tasks.

FlashRT achieves up to 70x latency reduction in multimodal applications by intelligently guiding coding agents through a multi-pass optimization process.

TPIPS reveals that traditional metrics fail to capture the nuanced aspects of visual similarity, leading to a performance gap that can be bridged with context-aware evaluation.

A training-free anomaly detection framework that tracks object transformations in industrial videos outperforms existing methods while providing interpretable insights.

Automatic music transcription models struggle with pop music, as evidenced by a mere 38% Onset F1 score on the new MulTTiPop dataset.

Multimodal unlearning could revolutionize how we handle sensitive data in AI, enabling targeted removal without sacrificing model performance.

Achieving 80% throughput with Classifier-Free Guidance challenges the assumption that CFG drastically reduces efficiency in multimodal audio processing.

MoVA achieves superior video-text alignment by disentangling evolving visual concepts from static textual descriptions, outperforming existing models in handling long sequences.

Even top-performing AI models struggle with PowerPoint tasks, achieving only 45% success rates despite a robust evaluation framework that rewards nuanced performance.

ELASTIC reduces wall-clock latency by 34% while matching the success rates of the best-performing models in real-world robot manipulation.

HTT enables tactile learning across diverse sensors, achieving adaptability that was previously unattainable in contact-rich manipulation tasks.

Touch is not just an add-on; it fundamentally enhances object representation, leading to dramatic improvements in physical property estimation and manipulation tasks.

Current multimodal conversational models miss critical emotional cues, revealing a significant gap in their ability to engage in nuanced human-like dialogue.

Lift4D achieves unprecedented accuracy in 4D reconstruction of dynamic objects, even in the presence of severe occlusions and complex motions.

Local names boost retrieval accuracy, but models still fail to generate images that faithfully represent specific street segments.

Instruction blindness in VLA models can be mitigated by optimizing for flatter loss landscapes, leading to over 60% better adherence to language instructions.

Rule-grounded reasoning can cut average distance errors in driving VLAs by nearly half, fundamentally enhancing their decision-making transparency and reliability.

T2I models can be effectively probed for identity memorization without any access to training data, revealing surprising differences in how they handle famous versus less recognized names.

LLMs can outperform humans in predicting the next speaker in meetings, even without audio or visual data.

Dixtral achieves up to 29% absolute improvement in speaker-attributed transcription accuracy by leveraging diarization masks without risking catastrophic forgetting.

TuneJury achieves superior music preference alignment with a single frozen reward model that adapts efficiently to new audio generators.

Humanoid robots can learn to distinguish themselves from others purely through proprioceptive-visual cues, enabling advanced social interactions without predefined identities.

Reducing inter-utterance silence from 9.6 seconds to 0.3 seconds transforms the quality of real-time game commentary, making it feel more natural and engaging.

Joint image-depth generation can be achieved with a single model trained on sparse data, outperforming existing methods by a significant margin.

Unstructured human videos can unlock scalable robot skill acquisition, enabling zero-shot transfer across diverse tasks with minimal supervision.

Behavioral INR reveals that self-supervised learning can effectively disentangle complex, overlapping policies from unlabeled behavioral data, outperforming traditional methods in high-dimensional settings.

A conformalized language feedback policy can boost VLA performance by over 65% while ensuring safe and reliable task execution in novel environments.

Achieving robust sim-to-real transfer, this framework allows robots to learn dexterous skills directly from human videos, outperforming traditional methods.

AHEAD transforms static VLA models into dynamic manipulators, achieving up to 97% success in scenarios where traditional methods fail spectacularly.

Navigating with fewer than 8 VLM calls per episode, Goal2Pixel redefines efficiency in vision-language navigation tasks.

Imagine generating photorealistic 3D head avatars from a single 2D image, no multi-view data needed – MVCHead makes it a reality.

UMMs struggle with cross-modal consistency not from a lack of shared representations, but from misaligned latent space transformations, which LatentUMM fixes.

VLMs can be significantly boosted on embodied tasks by mid-training on a carefully curated subset of VLM data that is highly aligned with the VLA domain, rivaling the performance of much larger models.