Search papers, labs, and topics across Lattice.
This paper introduces a novel approach to sketch generation by predicting a 2D field that determines the order of stroke application, rather than generating strokes sequentially. Utilizing a pretrained latent flow-matching transformer, the authors train a VAE's decoder to generate an intermediate representation that includes the stroke order field, stroke mask, and segmentation. The resulting model can create ordered vector sketches from text descriptions or convert images into ordered vector representations, adhering to specified drawing instructions.
By reversing the conventional stroke generation process, this method allows for more intuitive and context-aware sketching directly from text prompts.
We invert the typical formulation of sketch generation: instead of drawing strokes in order, we predict a 2D field that defines the order in which strokes are drawn. We use a pretrained latent flow-matching transformer to supply the image prior to predict an intermediate representation, while training the VAE's decoder to predict the order field, stroke mask, and stroke segmentation. We vectorize the predicted segmentation into polylines and sort them by the field, producing an ordered vector sketch. Our model can predict an ordered vector sketch from a text description or derender an image into ordered vectors; for either, it follows text instructions specifying the order of drawing.
