Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
To overcome the inability of weight-code bounds to account for runtime activation and key-value cache precision, this work develops Behavioral Capacity Certificates (BCC), which measure quantized model complexity using the aggregate prior mass of functionally equivalent implementations that preserve bounded loss. The authors leverage this behavioral formulation to establish a forward-only mixed-precision screening pipeline and identify margin-certified weight cells that permit combinatorial pruning and sign-flipping without altering predictions. Evaluated on OLMoE, Qwen2.5, and SmolLM2, BCC delivers nonvacuous population loss bounds for complete autoregressive decoders and demonstrates that allocating higher bit-precision to keys than values consistently optimizes both perplexity and complexity certificates under fixed cache budgets.
Allocating more precision bits to keys than values consistently beats symmetric KV caching across perplexity, prediction agreement, and generalization bounds, breaking through standard weight-only compression theory.
Activation and key-value cache precision change what a quantized language model computes without altering its stored weights. Direct weight-code bounds, however, assign identical complexity to deployments that behave differently and charge separately for weight codes that behave identically. Behavioral Capacity Certificates (BCC) charge for behavior using the aggregate prior mass of complete implementations---weights, scales, activation and cache rules---that induce the same bounded loss. When quantization merges implementations, this shared mass lowers the complexity penalty, and a break-even law determines when the saving survives the cost of validating it. BCC supports a three-step deployment workflow, and our experiments verify each step. First, a forward-only screen shortlists per-layer bit-widths by how often candidate perturbations preserve the reference predictions, with quality comparable to Hessian-guided selection at lower preprocessing cost. Second, margin-certified cells identify weights that can be pruned or sign-flipped without changing the deployed behavior: every permitted combination preserves all declared predictions, and on OLMoE-1B-7B and SmolLM2-1.7B, independent probes bound the probability that any permitted combination changes a prediction on new text. Third, BCC bounds the population loss of the deployed model, nonvacuously for complete decoders and more tightly than the compressed-code route. At equal cache memory, giving keys higher precision than values yields lower NLL and higher prediction agreement on GPT-2, Qwen2.5, and SmolLM2, together with a tighter complexity bound in the GPT-2 audit.