Inference & Quantization - Weekly Roundup

Edge service orchestration can achieve up to 64.5% lower decision latency than LLMs by using Jev decision models, even at high loads where LLMs fail.

Experiments on multiple LLMs across different bit budgets show that CASA achieves lower perplexity than the latest scalar-proxy baselines, especially at ultra-low bit-widths ($<3$ bits per weight), confirming the distortion bound as a practical indicator of scalar-proxy failure.

This paper proposes a novel lightweight plastic-memory framework specifically designed for few-shot incremental learning on graphs, featuring an innovative evolving micro-clustering structure that dynamically updates representations of class prototypes, sub-prototypes, and their interaction weights.

STD is proposed, a hierarchical token pruning framework that adapts token selection mechanisms to the functional role of each network stage, and introduces a Stability-Adaptive Trigger in deep layers to execute pruning only during semantically stable phases.

RealFit is proposed, a framework that combines Unidirectional Information Flow (UIF) with Decoupled Timestep Modulation (DTM) that isolates the garment condition from stochastic noise to preserve garment identity, while DTM optimizes the modulation scale to maintain a strong conditional signal.

Embedding optical scattering priors directly into a lightweight neural pipeline yields a 10.8 dB PSNR gain over deep learning baselines under joint noise-scattering degradation, all within a 200 ms INT8 edge footprint.

This work introduces ACLArena, a framework for comprehensively studying, analyzing, and evaluating Agent Continual Learning, and proposes a new ACL recipe that combines offline replay over high-quality trajectories with a routed network of multiple LoRA experts each specialized via RL, substantially improving the agent's ability to learn across multiple domains.

Unalloc, an open-source tool that joins OpenCost, LiteLLM, OpenAI and Anthropic cost data into one exact ledger and reports the share of spend with no owner, is presented to study where attribution breaks at the seams between these systems.

This work introduces Harness-Zero, which enables harness distillation through agent-as-harness, and shows that for frontier LLMs using the same evolved harness, agent-as-harness outperforms code-as-harness.

An information-efficiency ratio (IER) based on a signal-to-noise decomposition and a candidate-set approximation enables token selection based on IER and its combination with existing usefulness scores, while retaining the sampled reverse-KL training objective.

WorldCrafter is a video world model that learns a camera-queryable implicit 3D-aware memory that enables streaming scene exploration from a single input image or text prompt and shows substantial gains in long-horizon consistency and camera-control accuracy while preserving visual quality during minute-scale exploration.

Experiments on three open-weight LLMs show that SimPrint reliably recovers the owner signature in both clean and modified settings, remains robust under fine-tuning, pruning, quantization, model merging, and serving-time perturbations, and maintains comparable downstream utility.

This work presents HOTICE, a whole-body humanoid learning framework for transporting objects through cluttered environments, and introduces Humanoid-Object Decoupled Potential Fields, which jointly encode collision-avoidance guidance for the robot and the carried object, enabling coordinated, obstacle-aware motion for both.

The results show that low-bit QAT, architectural slimming, FINN folding and FIFO optimization, and moderate clock scaling can jointly provide a practical real-time detector on a small Zynq FPGA.

SPECTRA is presented, a runtime-reconfigurable tiled architecture that sustains high utilization across the full speculative decoding pipeline and achieves up to up to $2.09\times$ speedup from tile-level reconfiguration and a further $1.25\times$ gain from system-level adaptability over fixed designs.

TinyCall is presented, a lightweight neural audio codec designed for real-time speech communication on low-power platforms such as the ESP32 microcontroller and Raspberry Pi and demonstrates that practical neural speech communication is feasible on highly resource-constrained edge devices.

Protein annotation doesn't have to be trapped in rigid ontologies: a 4-bit 3B LLM can generate curator-grade functional descriptions directly from raw amino acid sequences.

Edge service orchestration can achieve up to 64.5% lower decision latency than LLMs by using Jev decision models, even at high loads where LLMs fail.

AMPLE-Math, a reusable suite of 5,319 mathematical problems with six reasoning views that share the same answer, is constructed and AMPLE-Math, a reusable suite of 5,319 mathematical problems with six reasoning views that share the same answer, is compared, suggesting that OPSD can improve access to existing reasoning capabilities through parameters shared by direct-response and thinking-enabled inference.

This work instantiates VDN on MiniMax H3, which combines local Softmax attention with bidirectional linear memory for long-range video context and introduces Video Delta Attention (VDA), which updates memory once per frame by jointly incorporating its spatial tokens.

Using realistic communication models reversed simulated traffic speed trends and more than doubled predicted hard-braking events, so researchers and transportation agencies can maintain existing simulation pipelines while accurately accounting for dense-traffic packet loss and denial-of-service impacts.