Search papers, labs, and topics across Lattice.
This paper introduces StructuredEdit, a novel graphic design editing pipeline that utilizes Differentiable Parameter Propagation (DPP) to effectively incorporate hard design constraints into vision-language model fine-tuning. By reframing design editing as parameter manipulation rather than pixel generation, StructuredEdit achieves a significant increase in constraint satisfaction, reaching 89% compared to the 52% of existing models like GPT-4V. Additionally, user studies demonstrate that the new system reduces editing time by 33% and correction iterations by 44%, highlighting its practical advantages in professional workflows.
Achieving 89% constraint satisfaction in graphic design edits, StructuredEdit outperforms existing models while drastically reducing editing time and correction iterations.
Graphic design editing requires precise manipulation of typography, layout, and visual hierarchy under strict design constraints. Following the introduction of large language models, organizations have increasingly promoted vision-language models to enhance productivity. However, current models operate on pixels and achieve only 52% constraint satisfaction on structured design edits, thereby limiting their reliability for professional workflows. We present StructuredEdit, a pipeline that reframes design editing as parameter manipulation rather than pixel generation. Our core technical contribution is Differentiable Parameter Propagation (DPP), a training method that embeds hard design constraints into vision-language model fine-tuning by backpropagating pixel-level constraint violations through a lightweight differentiable rasterizer. A hybrid candidate-and-filter pipeline produces 125k validated edit triplets. The resulting system reaches 89% constraint satisfaction versus 52% for GPT-4V, 0.82 matched-element Intersection over Union, and 76% top-1 font accuracy over the 100 most-frequent design typefaces. In a user study (N=35), editing time drops 33% and correction iterations drop 44% relative to a GPT-4V baseline.
