Track papers across labs, topics, and trends.
Search papers, labs, and topics across Lattice.
This study introduces ManimTrainer, a novel training pipeline that integrates Supervised Fine-tuning (SFT) and Reinforcement Learning-based Group Relative Policy Optimization (GRPO) to enhance Large Language Models (LLMs) for generating programmatic animations with Manim. The research evaluates 17 open-source sub-30B LLMs, revealing that while SFT improves code quality, GRPO significantly boosts visual outputs and model responsiveness during inference. Notably, the Qwen 3 Coder 30B model, utilizing GRPO and an augmented inference strategy, achieved a 94% Render Success Rate and outperformed the baseline GPT-4.1 in visual similarity metrics, underscoring the effectiveness of the proposed methodologies.
A unified training and inference approach for LLMs in animation generation reveals that combining SFT and GRPO can dramatically enhance both code quality and visual fidelity.
Generating programmatic animation using libraries such as Manim presents unique challenges for Large Language Models (LLMs), requiring spatial reasoning, temporal sequencing, and familiarity with domain-specific APIs that are underrepresented in general pre-training data. A systematic study of how training and inference strategies interact in this setting is lacking in current research. This study introduces ManimTrainer, a training pipeline that combines Supervised Fine-tuning (SFT) with Reinforcement Learning (RL) based Group Relative Policy Optimisation (GRPO) using a unified reward signal that fuses code and visual assessment signals, and ManimAgent, an inference pipeline featuring Renderer-in-the-loop (RITL) and API documentation-augmented RITL (RITL-DOC) strategies. Using these techniques, this study presents the first unified training and inference study for text-to-code-to-video transformation with Manim. It evaluates 17 open-source sub-30B LLMs across nine combinations of training and inference strategies using ManimBench. Results show that SFT generally improves code quality, while GRPO enhances visual outputs and increases the models' responsiveness to extrinsic signals during self-correction at inference time. The Qwen 3 Coder 30B model with GRPO and RITL-DOC achieved the highest overall performance, with a 94% Render Success Rate (RSR) and 85.7% Visual Similarity (VS) to reference videos, surpassing the baseline GPT-4.1 model by +3 percentage points in VS. Additionally, the analysis shows that the correlation between code and visual metrics strengthens with SFT and GRPO but weakens with inference-time enhancements, highlighting the complementary roles of training and agentic inference strategies in Manim animation generation.