Search papers, labs, and topics across Lattice.
This paper introduces a test-time iterative optimization framework that transforms reference-consistent image generation from an open-loop to a closed-loop system, addressing the limitations of existing methods that lack feedback mechanisms. By employing a sensor-controller architecture with a modified PID algorithm, the framework iteratively adjusts latent control signals based on discrepancies between generated outputs and reference targets. The approach, which is training-free and model-agnostic, shows significant performance improvements over traditional open-loop methods, achieving up to 25.36% gains in facial similarity and substantial reductions in spatial errors for pose and depth consistency.
Transforming image generation into a dynamic feedback system yields up to 25.36% better facial similarity and significant spatial error reductions without any training.
While controllable image generation has made significant strides by incorporating visual reference conditions, existing methods predominantly operate as open-loop systems. They inject control signals in a strictly feed-forward manner, failing to guarantee strict fidelity to the reference due to the absence of active feedback and error correction mechanisms. To address this fundamental limitation, we propose a novel test-time iterative optimization framework that reformulates reference-consistent generation as a closed-loop dynamic tracking problem. By treating the pre-trained generative model as a control plant, our framework employs a sensor-controller architecture driven by a modified Proportional-Integral-Derivative (PID) algorithm. This mechanism iteratively optimizes the latent control signals at test time based on the sensed discrepancy between the generated output and the reference target. Notably, this approach is entirely training-free, model-agnostic, and integrates seamlessly around existing diffusion pipelines. Extensive evaluations across ID-preserving, pose-controlled, and depth-controlled generation tasks validate the universality of our method. Empirical results demonstrate improvements over computation-matched open-loop baselines, achieving relative performance gains of up to 25.36\% for facial similarity, alongside spatial error reductions of up to 27.71\% for pose alignment and 28.50\% for depth consistency. More broadly, this work offers a new conceptual perspective: it demonstrates that controllable generation can be effectively managed as a dynamic feedback system, bringing the rigorous principles of classical control theory into the optimization of generative models. Code is available at https://github.com/zzdrill/From-Open-Loop-to-Closed-Loop.
