Search papers, labs, and topics across Lattice.
The authors introduce DaTikZ-V4, a 4x larger and higher-quality dataset for text-to-TikZ generation, and train TikZilla, a family of small Qwen models (3B and 8B) using a two-stage pipeline of supervised fine-tuning (SFT) followed by reinforcement learning (RL). The RL stage uses an image encoder trained via inverse graphics to provide semantically faithful reward signals. Human evaluations demonstrate that TikZilla outperforms GPT-4o and matches GPT-5 in image-based evaluations, despite being much smaller.
A clever RL setup lets small, open-source models beat GPT-4o at generating scientific figures from text, proving that data quality and reward engineering can trump sheer size.
Large language models (LLMs) are increasingly used to assist scientists across diverse workflows. A key challenge is generating high-quality figures from textual descriptions, often represented as TikZ programs that can be rendered as scientific images. Prior research has proposed a variety of datasets and modeling approaches for this task. However, existing datasets for Text-to-TikZ are too small and noisy to capture the complexity of TikZ, causing mismatches between text and rendered figures. Moreover, prior approaches rely solely on supervised fine-tuning (SFT), which does not expose the model to the rendered semantics of the figure, often resulting in errors such as looping, irrelevant content, and incorrect spatial relations. To address these issues, we construct DaTikZ-V4, a dataset more than four times larger and substantially higher in quality than DaTikZ-V3, enriched with LLM-generated figure descriptions. Using this dataset, we train TikZilla, a family of small open-source Qwen models (3B and 8B) with a two-stage pipeline of SFT followed by reinforcement learning (RL). For RL, we leverage an image encoder trained via inverse graphics to provide semantically faithful reward signals. Extensive human evaluations with over 1,000 judgments show that TikZilla improves by 1.5-2 points over its base models on a 5-point scale, surpasses GPT-4o by 0.5 points, and matches GPT-5 in the image-based evaluation, while operating at much smaller model sizes. Code, data, and models will be made available.
