Search papers, labs, and topics across Lattice.
This paper introduces Instance-Specific Parametric Absorption (ISPA), a novel framework designed to enhance memory efficiency in autoregressive video generation by transitioning from Full-Attention to Local-Attention layers. By absorbing historical context into the model's weights rather than discarding it, ISPA effectively reduces the Key-Value cache size by up to 50% while maintaining near-lossless visual quality. The approach addresses the critical bottleneck of memory overload in streaming models, thereby improving inference throughput and long-range dependency retention in generated videos.
Memory overload in autoregressive video generation can be tackled by absorbing historical context into model weights, achieving up to 50% cache reduction with minimal quality loss.
Autoregressive (AR) streaming models have emerged as a powerful paradigm for long video generation. However, the linearly growing Key-Value (KV) cache poses a significant bottleneck, leading to memory overload and degraded inference throughput. A common compression method is to drop redundant KV tokens, which often breaks long-range dependencies, resulting in temporal flickering and identity loss. In this paper, we propose Instance-Specific Parametric Absorption (ISPA), a novel framework that shifts the KV cache compression from discarding to distilling. The core idea is to transit a subset of layers from Full-Attention (F-Layers) to memory-efficient Local-Attention (L-Layers) by "absorbing" historical context into the model's weights. Specifically, during a brief warmup phase, ISPA monitors the output discrepancy between global and local attention. At the transition point, we solve a closed-form least-squares problem to compute an instance-specific weight modulation that compensates for the missing history. Experiments across architectures (1.3B to 14B) demonstrate that ISPA can remove up to 50\% of the KV cache with near-lossless visual quality. We hope this perspective encourages future work to explore parametric memory consolidation beyond external token-level cache management for streaming generative models.
DAMO