Search papers, labs, and topics across Lattice.
This paper introduces MUSE, a novel approach that leverages the fixed sinusoidal timestep embedding in one-step diffusion models to serve as a task steering signal for monocular dense prediction. By eliminating the need for parameter-heavy adapters and learnable task tokens, MUSE achieves efficient multi-task learning while maintaining competitive performance across multiple datasets. The findings reveal that this method not only simplifies the architecture but also enhances the generalization capabilities of existing generative models like U-Net and DiT.
Unlocking the latent potential of timestep embeddings allows for a parameter-free, single-model approach to multi-task learning that rivals traditional, heavier methods.
Monocular dense prediction has recently seen remarkable success by repurposing pre-trained diffusion models. This opens a promising yet challenging avenue for more efficient multi-task learning paradigm. However, existing multi-task diffusion methods often introduce parameter-heavy adapters, experts, or learnable task tokens, leading to computational redundancy. In this paper, we reveal an inherent mechanism within one-step diffusion models: the native, fixed sinusoidal timestep embedding can be repurposed as an endogenous task steering signal. Based on this discovery, we propose Multi-task Unified eStimation via timestep Embedding (MUSE), a parameter-free, single-model multi-tasking approach for dense prediction. We interpret this mechanism via Manifold Decoupling, where discrete, fixed timestep values deterministically steer the generation process towards decoupled, task-specific manifolds in the latent space. Extensive experiments across 10 datasets demonstrate that MUSE achieves highly competitive performance on both monocular depth and normal estimation, and its efficacy generalizes across U-Net and DiT architectures. Our work offers a concise and efficient path toward generalist vision models by simply unlocking the latent potential of existing generation infrastructure.
