Jongse Park

This work presents the first systematic characterization of attention sparsity in autoregressive image generation across diverse workloads and representative open-source models and reveals several distinguishing properties, including a pronounced prefill-decode asymmetry, strong attention concentration on prompt and local tokens, and a unique diagonal attention sparsity pattern arising from the spatial locality of visual tokens.