OmniLottie: Generating Vector Animations via Parameterized Lottie Tokens

2026-03-02

AI Summary

This work addresses the structural redundancy of Lottie JSON files, which hinders their direct use for high-quality vector animation generation. To overcome this limitation, the authors propose a parameterized Lottie tokenizer that decomposes animations into structured command-parameter sequences and introduce the first multimodal instruction-driven framework for vector animation synthesis. By integrating Lottie format parsing, sequence modeling, and a pretrained vision-language model, the method enables precise mapping from text-image instructions to coherent animations. The study also releases MMLottie-2M, a large-scale, professional-grade dataset comprising over two million animation samples. Experimental results demonstrate that the generated animations exhibit strong semantic consistency, visual expressiveness, and high fidelity to user instructions, confirming the effectiveness and controllability of the proposed approach.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Language and Vision

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

OmniLottie is a versatile framework that generates high quality vector animations from multi-modal instructions. For flexible motion and visual content control, we focus on Lottie, a light weight JSON formatting for both shapes and animation behaviors representation. However, the raw Lottie JSON files contain extensive invariant structural metadata and formatting tokens, posing significant challenges for learning vector animation generation. Therefore, we introduce a well designed Lottie tokenizer that transforms JSON files into structured sequences of commands and parameters representing shapes, animation functions and control parameters. Such tokenizer enables us to build OmniLottie upon pretrained vision language models to follow multi-modal interleaved instructions and generate high quality vector animations. To further advance research in vector animation generation, we curate MMLottie-2M, a large scale dataset of professionally designed vector animations paired with textual and visual annotations. With extensive experiments, we validate that OmniLottie can produce vivid and semantically aligned vector animations that adhere closely to multi modal human instructions.

Problem

• vector animation
• Lottie
• multi-modal instruction
• animation generation
• structured representation

Innovation

• Lottie tokenizer
• vector animation generation
• multimodal instruction
• vision-language model
• structured sequence representation