See Before You Code: Learning Visual Priors for Spatially Aware Educational Animation Generation

2026-05-15

AI Summary

This work addresses the visual defects—such as element overlap, misalignment, and animation discontinuity—that commonly arise when large language models generate educational animation code without access to rendering feedback. To mitigate these issues, the authors propose OmniManim, a framework that incorporates a Vision Agent module to predict sparse keyframe layouts and enforces rendering-aware constraints during code generation. OmniManim leverages shared scene states, explicit visual planning, coarse-to-fine bounding box denoising, and interpolation-aware optimization to align generated code with perceptually coherent animations. The framework further integrates a multi-agent collaboration mechanism and structured rendering diagnostics to enable localized repairs. Evaluated on the EduRequire-500 dataset, OmniManim significantly outperforms both single-model and multi-agent baselines, with ablation studies confirming the critical contribution of each visual planning component.

Abstract

Large language models can generate executable code for educational animations, but the resulting renders often exhibit visual defects, including element overlap, misalignment, and broken animation continuity. These defects cannot be reliably detected from the code alone and become apparent only after execution. We formalize this problem as render-feedback-aware constrained code generation: given a natural language specification, the model must generate executable code whose rendered output satisfies structured quality criteria that can be evaluated only after rendering. To address this problem, we introduce OmniManim, a render-feedback-aware educational animation generation framework built around a shared scene state, explicit visual planning, structured post-render diagnostics, and localized repair. Within OmniManim, the Vision Agent is a task-specific visual planning module: it predicts sparse keyframe layouts with coarse-to-fine bounding-box denoising and optimizes an interpolation-aware objective to reduce intermediate-frame failures induced by downstream animation interpolation. We further construct two datasets, ManimLayout-1K and EduRequire-500, and provide a reproducible evaluation protocol covering executability, instructional quality, visual quality, and efficiency. On EduRequire-500, OmniManim improves measured render quality over both single-model baselines and existing multi-agent frameworks. Systematic ablation studies further verify that explicit visual planning, especially its coarse spatial prior, bounding-box refinement, and interpolation-aware optimization, is central to these gains.

Problem

• render-feedback-aware
• visual defects
• educational animation
• constrained code generation
• spatial awareness

Innovation

• render-feedback-aware
• visual planning
• spatial prior
• animation interpolation
• constrained code generation