Search papers, labs, and topics across Lattice.
This paper introduces ManimAgent, a self-evolving multimodal agent designed to enhance code generation for mathematical animations by utilizing a dual-channel Episodic Memory Bank that retains reflection experiences across tasks. By leveraging a vision-language model to evaluate rendered keyframes, the agent effectively stores successful strategies and failure patterns, enabling it to learn from past experiences without requiring weight updates or human intervention. The results show that as the memory size increases, the agent's performance improves significantly, evidenced by higher blind human Pass@1 scores and reduced reflection rounds compared to baseline methods.
Memory-augmented agents can learn from past failures and successes across tasks, leading to significant performance improvements in code generation for visual education.
Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode: lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code-generation task: from a scientific paper section, the agent writes Python in the open-source Manim library to render a mathematical animation. We present ManimAgent, a self-evolving multimodal agent that carries reflection experience across tasks through a dual-channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision-language model scores the rendered keyframes; the resulting signals populate a positive channel M+ that stores success rationales as soft Reference Examples, and a negative channel M- that stores validated failure patterns as hard Known Pitfalls. On a fixed-probe evaluation against no-memory, matched-budget retrieval-augmented generation, and shuffled-memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows. We will release the code, frozen memory snapshots, and the task stream.
