MUSE: Unlocking Timestep as Native Task Steering for One-Step Dense Prediction

2026-06-29

AI Summary

This work addresses the computational redundancy in existing single-step diffusion models for multitask dense prediction, which typically rely on parameter-heavy adapters or learnable task tokens. The study is the first to reveal and exploit the fixed sinusoidal timestep embeddings inherent in diffusion models as endogenous task-conditioning signals, proposing a unified multitask learning paradigm that requires no additional parameters. Built upon pretrained diffusion models, the method leverages timestep embeddings for task guidance and incorporates manifold disentanglement to enable task-specific generation, compatible with both U-Net and DiT architectures. Experiments across ten datasets demonstrate that the approach achieves performance on par with state-of-the-art methods in monocular depth and surface normal estimation, confirming its effectiveness and broad applicability.

Abstract

Monocular dense prediction has recently seen remarkable success by repurposing pre-trained diffusion models. This opens a promising yet challenging avenue for more efficient multi-task learning paradigm. However, existing multi-task diffusion methods often introduce parameter-heavy adapters, experts, or learnable task tokens, leading to computational redundancy. In this paper, we reveal an inherent mechanism within one-step diffusion models: the native, fixed sinusoidal timestep embedding can be repurposed as an endogenous task steering signal. Based on this discovery, we propose Multi-task Unified eStimation via timestep Embedding (MUSE), a parameter-free, single-model multi-tasking approach for dense prediction. We interpret this mechanism via Manifold Decoupling, where discrete, fixed timestep values deterministically steer the generation process towards decoupled, task-specific manifolds in the latent space. Extensive experiments across 10 datasets demonstrate that MUSE achieves highly competitive performance on both monocular depth and normal estimation, and its efficacy generalizes across U-Net and DiT architectures. Our work offers a concise and efficient path toward generalist vision models by simply unlocking the latent potential of existing generation infrastructure.

Problem

• multi-task learning
• diffusion models
• dense prediction
• parameter efficiency
• monocular vision

Innovation

• timestep embedding
• parameter-free multitasking
• one-step diffusion
• manifold decoupling
• dense prediction