Training Efficiency & Optimization

Targeting Super Weights in LLMs can lead to performance collapse, challenging assumptions about parameter importance and trainability.

By isolating the double intractability of expected information gain, this method slashes the computational costs of training adaptive policies in Bayesian experimental design.

Vanilla SGD with momentum struggles under heavy-tailed noise, revealing critical limitations that challenge its widespread use in optimization.

JEPA-style predictive learning can yield remarkably accurate network representations, achieving over 92% accuracy in classifying protocol families from partial data.

Achieving structured pruning that rivals unstructured methods in accuracy while significantly accelerating inference speed could redefine efficiency benchmarks for large language models.

Low-rank adaptation in vision-language alignment not only cuts costs but also boosts performance, revealing a surprising shift from hallucination to conservatism in model behavior.

SLORR achieves substantial model compressibility with under 1% training overhead, outperforming traditional regularization methods in preserving performance.

Relaxed speculative decoding can significantly boost sampling speed, but it comes with hidden costs in capability evaluation and model quality.

The stability of Extreme Learning Machines hinges on the hidden layer's singular value structure, revealing that SVD methods are crucial for reliable performance under challenging conditions.

Architecture-specific learning rate schedulers can boost model accuracy by over 6% compared to basic decay strategies, revealing a critical factor in neural network training success.

A simple Monte Carlo method can effectively train deep neural networks without relying on gradients, revealing surprising redundancies in their architectures.

Flat minima form a fiber bundle over spheres, revealing new structural insights that could transform our approach to optimizing deep learning models.

Robustness in neural networks can be quantified through new geometric insights, revealing polynomial bounds that could enhance classifier stability.

UltraX achieves the highest average performance across datasets while using fewer training tokens, redefining efficiency in data refinement for LLMs.

Reinforcement learning can drastically cut retraining costs in O-RAN without sacrificing performance, challenging traditional methods that rely on costly retraining.

FedOPAL achieves state-of-the-art accuracy in federated learning without incurring server-side training costs, revolutionizing edge intelligence collaboration.

ZipDepth achieves real-time monocular depth estimation on resource-constrained devices while rivaling the accuracy of much larger models.

Hidden Decoding achieves unprecedented performance improvements in large language models by scaling computation along the sequence length without modifying the Transformer architecture.

Small language models can achieve near state-of-the-art Text-to-SQL performance with just a fraction of the computational resources required by large models.

Accelerating MIONet training with a novel hybrid LSGD method could redefine efficiency benchmarks in deep learning architectures.

StepFM reveals that simple step data can outperform complex sensor models in predicting a wide range of health risks, making health monitoring more accessible and privacy-friendly.

Legacy paper ECGs can now be transformed into actionable diagnostic tools in under 30 seconds, even in resource-constrained settings.

Targeted layer insertion based on rigorous error estimation leads to superior generalization in neural networks, outperforming traditional architecture adaptation techniques.

Fast transductive rates in semi-supervised learning can be achieved with fewer labels than previously thought, thanks to the power of data augmentation.

Achieving state-of-the-art TSC performance without real data, TimEE redefines the potential of synthetic pre-training in classification tasks.

Full positive-definite geometry can precisely express descent directions, reshaping our understanding of optimizer effectiveness in gradient-based methods.

Achieving a target condition number in optimization may hinge on a new geometric framework that redefines preconditioning as a distance problem.

The way training signals are allocated between weight and bias pathways can fundamentally alter the optimization dynamics and generalization of neural networks.

PGA-DPS outperforms traditional sampling methods by integrating dataset priors and group sampling, leading to superior optimization in real-world applications.

Unifying diverse mathematical frameworks reveals critical insights into convergence and performance guarantees for reinforcement learning algorithms.

LoCA achieves state-of-the-art performance in vision tasks while preserving spatial priors, revolutionizing how we adapt convolutional models without full fine-tuning.

Unbounded Positive Asymmetric Optimization unleashes the full exploration potential of RL algorithms without sacrificing stability, revolutionizing how we train large language models.

STRACE transforms noisy execution traces into precise optimization signals, leading to a 42.5% to 58.5% success rate improvement in agent performance.

Adaptive prefix control can more than double the accuracy of GRPO on hard reasoning tasks while cutting trace length in half.

Tailoring sparsity to layer importance can slash perplexity by over 1.9 points, challenging the one-size-fits-all approach in transformer pruning.

DeLS-Spec achieves faster inference and longer acceptance lengths by decoupling long and short context predictions, all while slashing training costs.

TF-Engram achieves a notable performance boost in LLMs by integrating scalable, train-free semantic memory without the typical overhead of retraining.

Achieving 99.60% accuracy with a model that requires only 2,370 FLOPS could redefine the landscape of IoT security solutions.

Achieving up to 6× greater sample efficiency in diffusion RLHF by strategically reweighting timesteps and reusing informative trajectories could revolutionize how we align generative models with human preferences.

Transforming gradients into a near-isotropic space can cut LLM pretraining time by 7.6% while enhancing downstream task performance.

RNC-LM achieves a 34-fold speedup in potential-energy-surface fitting while maintaining robust convergence in complex optimization scenarios.

Discrete audio tokens can rival traditional spectral features in speaker verification when guided by a robust knowledge distillation framework.

Verifiable environments can empower web agents to self-evolve, achieving competitive performance without the need for external teacher models.

Single-rollout sampling can dramatically improve the stability and effectiveness of RL training for large language models, outperforming traditional methods by a significant margin.

MoWorld achieves real-time interactive performance on low-cost hardware, revolutionizing the deployment of World Models in practical applications.

LingBot-VLA 2.0 showcases a remarkable leap in robotic manipulation, achieving strong cross-embodiment performance with enhanced predictive capabilities.

TurnOPD redefines on-policy distillation by optimizing training budgets at the turn level, leading to superior agent performance without increasing training time.

Excluding low-loss observations during backpropagation can save up to 54% in compute while maintaining near-optimal model performance.

ActionCache can accelerate inference for VLA models by up to 34.43× without compromising task success rates, revolutionizing real-time robotic manipulation.

Estimation, not grid search, is key to optimizing LLM serving—this new approach reveals hidden performance potential in resource management.