Distributed Systems & Hardware - Weekly Roundup

Exploit sparse derivative structure to slash AD passes for Jacobians and Hessians, offering drop-in replacements for JAX users.

Token-level LLM routing, previously hindered by system inefficiencies, now achieves up to 64x higher throughput with a novel request-centric, model-centric serving architecture.

Training higher-order Topological Neural Networks on massive graphs like Reddit is now possible, thanks to a novel clustering-based mini-batching strategy that slashes GPU memory requirements by over 80%.

Heterogeneous experts can be dynamically coordinated via MoE gating to form composite model services, outperforming monolithic models and standard MoE in cost-aware marketplaces.

Gradient clipping, often seen as a heuristic, unlocks optimal convergence and linear agent speed-up for decentralized SGD under heavy-tailed noise, outperforming normalization in this regime.

Token-level LLM routing, previously hindered by system inefficiencies, now achieves up to 64x higher throughput with a novel request-centric, model-centric serving architecture.

Exploit sparse derivative structure to slash AD passes for Jacobians and Hessians, offering drop-in replacements for JAX users.

Training higher-order Topological Neural Networks on massive graphs like Reddit is now possible, thanks to a novel clustering-based mini-batching strategy that slashes GPU memory requirements by over 80%.

Heterogeneous experts can be dynamically coordinated via MoE gating to form composite model services, outperforming monolithic models and standard MoE in cost-aware marketplaces.

Gradient clipping, often seen as a heuristic, unlocks optimal convergence and linear agent speed-up for decentralized SGD under heavy-tailed noise, outperforming normalization in this regime.

Geometric weighting by update norm offers a subtle but measurable improvement in personalized federated learning, outperforming uniform aggregation and FedPer-R1 on CIFAR-10's native client accuracy.

Average-reward RL for general multichain MDPs is now possible without model knowledge or reduction to discounted problems.

LLMs develop inherent fault tolerance as they scale, learning to compute within error-correcting codes that remain efficient even for massive models.

Trillion-parameter agentic RL refits are now 12-40x faster, shrinking an 87.5-minute full checkpoint transfer to 150 seconds by sending only bit-exact weight deltas.

Parallel LLM agents can finally achieve near-linear speedups by strategically parallelizing only when token-based cost estimates predict a net gain, eliminating redundant work and reconciliation overhead.

Privacy-preserving federated learning achieves segmentation performance on skin lesions comparable to centralized methods, without centralizing sensitive patient data.

Collaborative edge-cloud LLM inference can achieve 275% higher throughput by dynamically synchronizing heterogeneous requests to batch their common cloud-side suffix.

Decentralized learning topologies can be optimized using only local neighborhood and friend-of-a-friend information, achieving competitive performance without global graph awareness.

Distributed MMD with Random Fourier Features achieves near-perfect correlation with expected drift magnitude on massive e-commerce data, while per-dimension KS tests falter.

Private prediction can achieve logarithmic dependence on the time horizon, while private learning requires polynomial dependence, revealing a fundamental gap in their sample complexity for differentially private settings.

Hierarchical MoE routing with a novel arrival-weighted mean estimation prevents expert collapse and enables localized co-activation, outperforming flat routing on large embedding models.

Physics-based secure key distribution can now sustainably support federated learning for medical imaging, reducing key material needs by 35x without sacrificing accuracy.