Training Efficiency & Optimization

Image reconstruction and generation quality rivaling dense tokenizers are achieved with ~130 tokens by adaptively refining global 1D representations with sparse 2D tokens.

Task-specific training, not just model scale, unlocks effective relative policy difference transfer in distillation, and the reference policy is key to isolating residual capabilities.

Embodied agents improve on complex tasks by recombining learned skills, but their ability to extract value from additional experience saturates, failing to scale with learning samples.

Slash LLM evaluation costs by over 270x without sacrificing accuracy, revealing that even top open-weight models struggle to outperform simple baselines on AML transaction fraud detection.

LLM distillation can achieve 18.55% higher math accuracy and 13.8 points better ALFWorld success by intelligently branching student trajectories at uncertain decision points, not just at turn starts.

Reasoning supervision can be compressed over 450x with uncertainty-guided contraction, boosting LLM efficiency without sacrificing capability.

Batching future MH proposals unlocks significant speedups for LLM power sampling without altering the core reasoning process.

Small models can now learn new skills without forgetting old ones, thanks to a novel distillation technique that leverages a larger model to guide the learning process.

One million parallel environments are insufficient without intelligent sampling: Success Guided Sampling unlocks mega-scale RL for robots by focusing training on the edge of learned capabilities.

Abstracting human demonstrations into scene graphs unlocks zero-shot generalization for dexterous manipulation policies, achieving 71% better performance on unseen tasks than imitation-heavy methods.

A single recurrent Transformer block can match the accuracy of deep vision encoders with 70% fewer parameters by dynamically programming FFN experts via a continuous depth coordinate.

Policy optimization for adversarial MDPs now matches state-of-the-art regret bounds by stabilizing local updates with regularized Q-functions, eliminating the horizon gap.

Source-side training dynamics, not just current values or training time, prospectively predict OOD degradation, even across model architectures.

Exploit sparse derivative structure to slash AD passes for Jacobians and Hessians, offering drop-in replacements for JAX users.

A single regional weather model can now leverage multiple global forecast drivers without retraining, achieving kilometer-scale accuracy.

Optimal oracle complexity for nonsmooth nonconvex composite optimization is finally achieved, matching noncomposite rates by cleverly incorporating the convex regularizer into online learning losses.

LLM value caches are surprisingly redundant, allowing for significant memory compression through simple symmetry exploitation without hurting inference speed.

Unifying density ratio estimation and distribution transport, this work reveals how a single displacement field can simultaneously capture statistical shifts and dynamical transformations between probability distributions.

Quantum classifiers grow to unpredictable depths regardless of training data size, and their generalization is not explained by active gate count alone.

Input-specific parameter generation unlocks adaptive time-series forecasting, outperforming static adaptation and improving zero-shot generalization.

Training higher-order Topological Neural Networks on massive graphs like Reddit is now possible, thanks to a novel clustering-based mini-batching strategy that slashes GPU memory requirements by over 80%.

Rare concepts in LLM fine-tuning are significantly hindered by "prior barriers" that require targeted supervision, not just more data.

Noise in Gaussian regression on Boolean cube polynomials can necessitate an exponential increase in sample complexity, requiring $(m+t)4^k\exp\{-O(k^{1/3})\}$ samples for parametric rates, a stark contrast to the $O((m+t)2^k)$ needed for noiseless identification.

Cloud workload forecasting models can now achieve near-optimal accuracy with only 20% of the compute cost of naive online learning by adaptively updating only when drift is detected.

Most published TTA methods for time-series forecasting are outperformed by simple, hyperparameter-free baselines under realistic label delays, with some even becoming harmful.

Reverse-mode gradients slash Bayesian inference time for expensive simulators, especially when parameters vastly outnumber outputs.

Spatio-temporal Bayesian optimization for point-process data is now scalable and efficient, achieving accurate peak discovery with minimal evaluations.

State-preserving constraints in Bayesian optimization are transformed into an efficient, anisotropic ellipsoid in design space, enabling robust optimization of complex engineering systems.

Reasoning models can learn to plan by reverse-engineering solution ideas from hindsight, even when initially unable to solve the problem.

LLMs can now autonomously learn to self-improve at test-time, nearly doubling utility by intelligently selecting and applying TTT strategies.

A training-free change detection method achieves near-optimal performance by integrating diffusion models to precisely estimate Hyvärinen scores, slashing detection delays by over 90% in simulations.

Weisfeiler-Lehman graph kernels unlock superior structural features for algorithm selection, outperforming traditional methods on challenging constraint satisfaction problems.

Gradient-based learning is now possible for variational sequential Monte Carlo without iterative solvers, thanks to a differentiable relaxation of systematic resampling.

QNN gate selection and arrangement, not just qubit count, critically dictate financial forecasting performance, with the best configuration outperforming classical BiLSTM and exhibiting significant noise sensitivity on current quantum hardware.

Random features and projections unlock efficient quadratic entropy estimation for complex distributions and distances, revealing nuanced interdisciplinary research landscapes.

End-to-end diffusion models can achieve state-of-the-art precipitation nowcasting with over 17x faster inference by forecasting directly in radar space.

Optimal regret in adversarial MDPs with stochastic hard constraints is finally achieved, bridging the gap between existing upper and lower bounds by achieving $\widetilde{\mathcal{O}}(\sqrt{T}/\rho + 1/(d\rho))$ regret.

Iterative refinement in data space, not flow space, unlocks fast, high-fidelity generative sampling without manual timestep tuning.

Contextual contamination in few-shot vision-language learning is mitigated by a novel semantic purification and reinforcement-learning gating mechanism, unlocking superior performance.

KL regularization's failure in group policy optimization stems from subtle reward interactions, but a new calibration method unlocks performance gains.

Interpretable archetypes emerge from discrete data using a principled likelihood framework, unlocking insights in genomics and microbiome analysis previously inaccessible to standard AA.

A reverse Hebbian-inspired pruning method achieves 99% sparsity on ResNet-18 with only a 2.4% accuracy drop, dynamically allocating sparsity without layer-specific targets.

Dynamic critic pessimism, guided by geometric disagreement, unlocks superior locomotion learning and efficiency in off-policy actor-critic methods.

Heterogeneous experts can be dynamically coordinated via MoE gating to form composite model services, outperforming monolithic models and standard MoE in cost-aware marketplaces.

Adversarial outliers can be effectively handled in linear dynamical system identification by leveraging group-sparsity and robust estimation techniques, achieving provable guarantees.

Spectral analysis reveals that increasing latent dimensions in Koopman autoencoders improves dynamics learning by concentrating learned eigenvalues in low-residual regions, not just by reducing one-step prediction error.

Structured linear dynamical systems can be learned from drastically undersampled observations ($p=o(1)$) with trajectory lengths far shorter than previously thought possible.

Pruning LLMs for decoding speedup is significantly improved by calibrating sparsity with actual generation activations, not just static text, and optimizing SpMV kernels.

Out-of-distribution data can dramatically boost discrete diffusion model performance under extreme data scarcity by strategically injecting it at low noise levels.

LLMs adapt neural operator configurations using both a strong task-dependent prior and real-time experimental feedback, outperforming Bayesian optimization.