Inference & Quantization

A novel Kronecker Delta Rule enables associative memories to revise targeted information with minimal interference, unlocking stable, long-context linear attention.

A simple two-layer transformer with $O(\log\log N)$-bit precision can perfectly copy arbitrary strings up to length $N$ and terminate correctly, a feat impossible for standard transformers with finite-precision softmax.

Lifelong agents forget up to 40% of context under resource constraints, but learning memory worth unlocks significant performance gains.

Decentralized models can learn to route queries to specialized neighbors at inference time, unlocking significant performance gains without a central orchestrator.

Neural posterior estimation's overconfidence problem is solved by splitting the optimization of mean and variance, yielding better out-of-sample performance.

Polynomial gradient orthogonalization achieves 15-18% faster runtimes than state-of-the-art methods by leveraging optimized polynomial circuits.

Near-autoregressive quality is achievable with parallel decoding, unlocking significant inference speedups across model scales.

Model merging is revealed as an optimal, closed-form stage of distillation, enabling efficient foundation model consolidation.

Adapting MoE experts directly, not through bypass adapters, unlocks the best downstream performance with minimal general capability loss.

Sparse attention in video diffusion models can be corrected using cached dense refinements, recovering fidelity without altering the reuse schedule.

Self-distillation in CTR prediction simplifies to a logit-averaged loss plus a disagreement penalty, enabling sample-free training and boosting single-tower accuracy.

Iterative GNNs trained on solver trajectories outperform single-pass SOCP solvers and enable warm-starting classical optimizers.

Adversarial attacks create provable geometric inconsistencies across prime-quantized and CNN embedding spaces, enabling theoretically guaranteed detection.

A novel minimax game formulation for multi-teacher distillation dynamically allocates optimization budget to ensure equitable progress across domains, even those with slower learning rates.

Agentic AI inference can be accelerated by 2-6x with history-free speculative decoding that eliminates KV cache overhead and boosts acceptance rates by up to 70%.

Remote KV caches can cripple LLM prefill latency, but sharding the KV store across parallel ranks and enabling asynchronous loading slashes p99 time from 83s to under 1s.

A strong LLM can review grounded drafts from a weak model, retaining most task performance while halving latency and memory traffic.

Distilling explicit, reward-guided skills into VLMs without the student model ever seeing the skills themselves unlocks superior multimodal performance.

Spatial memory via frustum slab routing slashes video world model inference time by over 4x without sacrificing generation quality.

Gating policy distillation with directional agreement between early and late RL shifts unlocks more effective knowledge transfer, boosting LLM performance on reasoning tasks.

Lossless compression of sparse ReLU activations slashes LLM activation VRAM by up to 31% with near-zero overhead.

Transformers trained with ABNAT learn to mimic clean attention maps, achieving up to 85.77% less attention deviation and 11.28% higher accuracy on noisy analog hardware.

Frozen LLMs reject sender KV caches even when geometrically similar, failing due to competitive attention, not fidelity.

Deep learning compilers are silently introducing bugs through state retention, with 35 stateful issues found in just four frameworks.

A simple, fixed-budget best-arm identification algorithm can be made nearly as adaptive as state-of-the-art methods by optimizing its phase schedule, achieving $11/12$ of the optimal normalized rate for Gaussian arms.

A two-bit code and shared reconstruction program slash LLM conditional memory storage by 75%, enabling 4x larger tables on GPU.

A 32.5x smaller grid foundation model achieves superior reactive power prediction and significant active power improvements over its larger teacher, enabling real-time edge inference.

Non-monotonic and fine-granular quantized activations are now efficiently implementable on custom DNN accelerators, unlocking broader hardware optimization.

Byte-level LLMs can achieve a 25% jump in math accuracy by patching where text is *new*, not just surprising.

Sharing LLM internal states is efficient, but only if you augment the KV cache and replace activations, not swap them.

Neural audio watermarking detectors can now reliably detect legacy content even after updates, while simultaneously rejecting unauthorized embedders, by explicitly training for angular separation in the embedding space.

Consolidating specialist LLMs into a single, more accurate generalist is possible with *zero* gradient updates, relying solely on forward passes and input statistics.

Learned token visibility control dramatically boosts LLM reasoning accuracy, achieving state-of-the-art performance on GSM8K and MATH-500 with a tiny, frozen inference-time policy.

Train LLMs with up to $1.455\times$ less memory and enable $3.5\times$ longer context lengths by losslessly compressing tensors on-the-fly.

A frozen, 16MB vocabulary can match or beat learned embeddings in small LLMs, unlocking dramatic speedups and memory reductions for edge deployment.

Aggressive KV cache compression can be achieved with near-zero accuracy loss by treating the cache as a trainable neural network optimized at test-time.

A simple, fixed query rule improves FID by over 2 points on CIFAR-10 without retraining, revealing prediction horizons are state-sensitive inference controls.

A single, stage-specific objective per expert in a decoupled diffusion distillation framework enables state-of-the-art video generation with a 10,000x reduction in training cost.

Teachers can be coaxed into recovering from their own errors during distillation, unlocking substantial gains in student model performance.

Distilling LLM agents by supervising semantic actions, not token sequences, unlocks significant performance gains and bypasses tokenization constraints.

Most self-distillation methods for agentic RL fail to consistently outperform a strong baseline, and impressive final scores often hide unstable training dynamics and substantial computational overhead.

Continuous integration over a piecewise linear memory curve unlocks exact normalized attention with analytical gradients, outperforming softmax and focal attention in causal Transformers.

A single, elastic transformer model can now dynamically adapt its compute budget across a continuum of sparsity levels, achieving up to 1.80x higher throughput and 1.26x lower latency on A100 GPUs without retraining.

A novel PPO variant leverages complementary temporal dynamics of self-teacher signals and critic value predictions to boost reasoning accuracy by up to 2.59pp.

LLM reasoning can be boosted at inference time by separately sharpening reasoning paths and final answers, outperforming existing power-sampling methods across multiple benchmarks.

Quantizing MoE models with mixed precision is significantly improved by calibrating expert error sensitivity using an information-theoretic contrast target, outperforming simple ranking-based methods.

Speculative decoding drafters can achieve over 2x speedup on 256K context by treating distant tokens as shared far memory with sharpened attention.

A simple reconstruction pre-training stage unlocks state-of-the-art one-step generative modeling with 10x less compute.

A closed-form, covariance-aware orthogonal transform for LLM quantization achieves state-of-the-art performance with a third of the memory and time of learned rotation methods.

Explicitly training LLM agents on runtime semantics via Harness Policy Distillation unlocks a 7.5x increase in autonomous task success and a 2.5x boost in runtime compliance on TRAJECT-Bench.