Track manipulation, locomotion and embodied systems.
Search papers, labs, and topics across Lattice.
To resolve the tension between target-domain photorealism and source feature preservation, the authors develop LocDiff, a zero-shot guidance framework for pre-trained diffusion models that selectively restricts guidance to critical spatial regions while leaving non-critical regions unconstrained. Standard global diffusion guidance over-regularizes background regions and distorts fine-grained semantic fidelity when bridging sharp domain shifts. Evaluated across zero-shot art-to-photo portrait translation and fetal ultrasound enhancement without ground-truth targets, LocDiff achieves a strictly superior realism-faithfulness trade-off compared to prior guided diffusion baselines.
Global diffusion guidance fails across large domain shifts by over-constraining the wrong regions; restricting sampling guidance strictly to identity-critical features unlocks zero-shot photorealistic domain adaptation without any model retraining.
Denoising Diffusion Probabilistic Models have shown remarkable performance in unconditional image generation. In order to generate images with desired semantics, recent works have restricted the solution space by using guidance constraints in the diffusion sampling process. However, for image enhancement across different domains, these methods struggle to balance two main requirements: looking realistic in the target domain (photorealistic images) and preserving relevant features of the source domain, e.g., low-quality renderings or art paintings. Here, small local changes can alter the fidelity of the image completely, while large changes in other regions might be insignificant. We introduce LocDiff, a locality-constrained guidance method for image enhancement, which serves as a zero-shot extension to pre-trained diffusion models, ensuring the preservation of critical features during domain adaptation. In this way, we retain important local features, while allowing less critical regions to remain unconstrained and not interfere with the guidance process for relevant regions. We evaluate our method on two different domain-shift tasks: For art-to-photo translation, we apply the method in a fully zero-shot setting, preserving facial identity from paintings while generating photorealistic details. For enhancing low-quality fetal ultrasound renderings, we demonstrate zero-shot inference with auxiliary prior alignment. Here, the objective is to artificially add high-resolution characteristics and produce photorealistic ultrasound renderings, a target domain for which no ground truth distribution exists. Our experimental results demonstrate that LocDiff achieves favorable realism-faithfulness trade-offs compared to state-of-the-art methods, enabling controllable cross-domain enhancement.