Search papers, labs, and topics across Lattice.
This paper introduces SVGEval, a novel framework for evaluating the perceptual quality of text-to-SVG generation by integrating visual renderings into the assessment process. The authors highlight the inadequacies of existing evaluation protocols, which often neglect SVG-specific qualities and rely on rasterized metrics, leading to a significant gap in performance on geometry and layout judgments. Systematic evaluations reveal that while models excel in semantic alignment and aesthetics, they struggle with spatial and geometric assessments, underscoring the need for a more nuanced evaluation approach.
Models may ace aesthetics but falter on geometry鈥擲VGEval reveals the critical gaps in text-to-SVG generation evaluation.
Multimodal large models are increasingly used to generate scalable vector graphics (SVG), but reliable evaluation remains underexplored. Existing protocols are often code-centric or borrow raster-image metrics after rendering SVGs, which fail to reflect human perception and overlook SVG-specific qualities such as geometry and spatial composition. We introduce SVGEval, a vision-grounded multimodal benchmark for human-aligned SVG quality assessment. SVGEval explicitly incorporates visual renderings to evaluate whether models can judge the rendered outcome rather than only inspect SVG code, and provides high-quality annotations obtained via multi-round human labeling with expert refinement. Systematic evaluations across representative multimodal models reveal a clear gap: models perform relatively well on semantic alignment and aesthetics, yet struggle on geometry- and layout-related judgments. Building on SVGEval, we train an explainable SVG quality scorer that outputs multi-aspect scores with textual rationales. Ablations show that explicit visual grounding and reasoning supervision are crucial, especially for spatial and geometric assessment. SVGEval offers a reliable testbed and practical scorer for evaluating and improving SVG generation in the era of multimodal models.
