Search papers, labs, and topics across Lattice.
This paper introduces SPARK, a novel input-conditioned sparse activation modulation technique for enhancing frozen Diffusion Transformer (DiT)-based super-resolution models. By leveraging dominant activation channels identified through an online ranking procedure, SPARK applies targeted affine transformations to a limited number of channels, enabling significant improvements in image fidelity and perceptual quality without the need for extensive fine-tuning. Experiments demonstrate that SPARK consistently outperforms traditional methods across multiple datasets while maintaining a lightweight architecture, suggesting a new pathway for efficient model adaptation in super-resolution tasks.
Targeted modulation of just eight dominant channels can dramatically enhance image super-resolution quality without fine-tuning the entire model.
Real-world image super-resolution (SR) increasingly relies on Diffusion Transformer (DiT) backbones, whose internal activations can be dominated by a small number of massive channels. Yet improving perceptual quality in these models still typically requires fine-tuning the network or attaching additional adapters, leaving this structured activation space largely unexplored for adaptation. We investigate whether dominant channels can instead serve as a compact adaptation interface for frozen DiT-based SR models. We first characterize their behavior in pretrained SR backbones and show through controlled interventions that they strongly affect reconstruction quality. Building on this observation, we introduce SPARK, a lightweight input-conditioned controller that predicts bounded per-channel affine transformations for only the selected channels, while keeping the SR backbone and VAE frozen. Dominant channels are identified through an online activation-ranking procedure, and only a small predictor conditioned on the low-resolution VAE latent is optimized. Experiments on three DiT-based SR backbones across DIV2K, RealSR, and DRealSR show consistent gains in both fidelity and perceptual quality while modulating only eight channels per stream and block. Controlled comparisons further show that these gains cannot be explained by parameter budget or access to the selected channels alone.
