ManimAgent: Self-Evolving Multimodal Agents for Visual Education

2026-06-29

AI Summary

This work addresses the challenge that existing large language model (LLM) agents struggle to retain and transfer reflective experiences across multi-turn tasks, often restarting from scratch. To overcome this limitation, the authors propose ManimAgent—a self-evolving multimodal agent that leverages a novel dual-channel episodic memory mechanism to continuously accumulate and transfer successful strategies as soft references and failure patterns as hard constraints, without requiring model weight updates or human intervention. The framework integrates an LLM, the Manim animation library, and a vision-language model into a retrieval-augmented generation system for automatically producing mathematical animations in scientific papers. Experiments demonstrate that ManimAgent significantly improves human-evaluated Pass@1 accuracy on a fixed test set, reduces the number of required reflection rounds, and exhibits steadily increasing performance as memory size grows.

Abstract

Multi-round reflection lets agents built on large language models recover from failures within a single task, but each task remains an isolated episode: lessons learned across many reflection rounds on one task are discarded before the next begins. We study this gap on a code-generation task: from a scientific paper section, the agent writes Python in the open-source Manim library to render a mathematical animation. We present ManimAgent, a self-evolving multimodal agent that carries reflection experience across tasks through a dual-channel Episodic Memory Bank grown entirely from its own task stream, with no weight updates and no human seeds. After each animation converges, a vision-language model scores the rendered keyframes; the resulting signals populate a positive channel M+ that stores success rationales as soft Reference Examples, and a negative channel M- that stores validated failure patterns as hard Known Pitfalls. On a fixed-probe evaluation against no-memory, matched-budget retrieval-augmented generation, and shuffled-memory baselines, blind human Pass@1 rises and reflection rounds fall as memory size grows. We will release the code, frozen memory snapshots, and the task stream.

Problem

• multimodal agents
• episodic memory
• cross-task learning
• reflection
• code generation

Innovation

• self-evolving agent
• episodic memory bank
• multimodal reflection
• Manim code generation
• cross-task learning