CMU Machine Learning

Larger RLVR batches, despite appearing efficient with square-root learning rates, suffer a critical batch size that degrades sample efficiency and hastens failure due to curvature effects.

Pretraining neural CFD surrogates across diverse domains unlocks substantial gains in zero- and few-shot generalization, drastically reducing data requirements for new applications.

Faster computer-use agents are bottlenecked by their speed, but existing benchmarks are irreproducible; this new standardized framework reveals counterintuitive speed-efficiency trade-offs and that current open-weight models lag behind.

Native 8-bit LLM training silently destabilizes at billion-parameter scale due to softmax gradient drift, but restoring the zero-row-sum invariant completely eliminates the FP8 attention gap without architecture modifications.

Looped flows are proposed, an approach that allows solving harder problems by spending more computation through a finer temporal grid and enables multiple valid predictions from different initial noise samples.

Achieving near state-of-the-art performance for under $7K opens the door for cost-effective language model training accessible to the broader research community.

GCA reduces communication overhead in federated learning by up to 99.15% while simultaneously enhancing data protection and improving model accuracy.

Explicitly optimizing for target function smoothness can dramatically enhance the performance of models on tabular data.

Cheap models can recover early evolutionary progress, enabling a shift in budget allocation that dramatically enhances LLM-driven algorithm discovery.

Path-level pretraining in MultiPathFormer leads to a dramatic 59% improvement in wireless propagation estimations, reshaping the landscape of wireless foundation models.

Retaining the right contextual information can boost long-context modeling performance by over 20% compared to traditional methods.

Achieving high-quality reranking with a 30B MoE model is now feasible on an academic budget, outperforming traditional dense models in efficiency.

Adapters may store less than half the information expected, with their capacity heavily influenced by parameter placement rather than quantity.

Estimating valid transport maps can be as hard as optimal transport, but under certain conditions, alternative maps can be learned with significantly higher accuracy.

S-JEPA sets a new standard in speech representation learning by achieving top performance with fewer parameters and without the cumbersome offline re-clustering process.

Recycling zero-variance queries during training can boost model performance, enabling a smaller 1.7B parameter model to match or exceed the accuracy of larger counterparts.

Gradient inference can revolutionize how we tackle gradient estimation in complex probabilistic programs, enabling new state-of-the-art estimators that outperform traditional methods.

Test-time performance can be significantly improved without a corresponding drop in validation loss, raising critical questions about model evaluation in deep learning.

Why pick just one token mixer when you can have them all, dynamically switching between attention and linear recurrences for optimal efficiency and performance?

Explicitly aligning MoE routing behavior during fine-tuning can significantly boost performance on multilingual tasks, especially when the model understands the task in English but struggles in the target language.

Combining spatial and temporal differences when training energy-based models unlocks better density estimation than using either alone.

LLMs can leverage "sleep" to distill long contexts into fast weights, unlocking superior reasoning without sacrificing inference latency.

UMMs struggle with cross-modal consistency not from a lack of shared representations, but from misaligned latent space transformations, which LatentUMM fixes.

Scale multi-agent RL diversity metrics to hundreds of agents without sacrificing accuracy: Graph-SND offers a drop-in replacement for quadratic SND calculations, achieving near-identical results with order-of-magnitude speedups.

Implicit time integration on GPUs gets a 3x speed boost thanks to a novel algebraic coarsening method that avoids costly explicit remeshing.

Attention bottlenecks in long-context decoding? SANTA slashes memory bandwidth demands by stochastically sampling value vectors, achieving 1.5x speedups without sacrificing accuracy.

VLMs can be significantly boosted on embodied tasks by mid-training on a carefully curated subset of VLM data that is highly aligned with the VLA domain, rivaling the performance of much larger models.

Dramatically improve multimodal recommendation accuracy without any training by initializing user embeddings with item modality features and user cluster information.

Forget full retraining: intelligently selecting data subsets using gradient-based representations can keep your generative recommender fresh and robust to drift.

Training speech separation models on real-world noisy data doesn't have to mean accepting noisy outputs: this method cuts residual noise in half.

LLMs learn skills in a surprisingly consistent order during pretraining, revealing a hidden curriculum that's predictable across models and readable from their internal representations.

Get more from less: SonoSelect intelligently guides ultrasound probes to achieve comparable diagnostic accuracy with far fewer views, slashing scanning time and processing costs.

Forget hand-picking your cross-lingual training data: a budget-constrained optimization can automatically allocate resources across multiple source languages, boosting performance on African languages by a large margin.

Expect to pay an exponential sample complexity price for computationally efficient mean and covariance estimation with missing data, but not for linear regression.

You can slash RoPE memory costs by 10x without sacrificing convergence, just by applying it to a sliver (10%) of hidden dimensions.

Flow matching's advantage in RL isn't distributional modeling, but rather its ability to correct value estimates iteratively and learn more adaptable features, leading to significant performance gains in challenging online settings.

Robots can now learn long-horizon tasks far more effectively by distilling complex histories into a few key visual moments, outperforming standard imitation learning by 70% on real-world tasks.

Variational learning can tame the inherent chaos of nanoscale devices, paving the way for practical, larger-scale probabilistic computers.