Snapmoji: Instant Generation of Animatable Dual-Stylized Avatars

2025-03-15

AI Summary

Existing personalized avatar systems face bottlenecks in expressiveness, customization efficiency, and rendering performance. This paper proposes Gaussian Domain Adaptation (GDA), the first method enabling single-selfie-driven, second-level generation of dual-style 3D Gaussian avatars. GDA synergistically integrates large-scale 3D geometric priors with 2D style transfer to achieve identity-preserving, two-level controllable stylization and dynamic facial expression transfer. Technically, it unifies 3D Gaussian splatting, GDA pretraining/fine-tuning, lightweight neural rendering, and end-to-end facial motion transfer. The system generates avatars in just 0.9 seconds and renders them at 30–40 FPS on mobile devices. Compared to state-of-the-art methods, our approach achieves significant improvements in expressiveness, generation speed, and cross-style generalization.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Multiagent Systems: Adversarial Agents
• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

The increasing popularity of personalized avatar systems, such as Snapchat Bitmojis and Apple Memojis, highlights the growing demand for digital self-representation. Despite their widespread use, existing avatar platforms face significant limitations, including restricted expressivity due to predefined assets, tedious customization processes, or inefficient rendering requirements. Addressing these shortcomings, we introduce Snapmoji, an avatar generation system that instantly creates animatable, dual-stylized avatars from a selfie. We propose Gaussian Domain Adaptation (GDA), which is pre-trained on large-scale Gaussian models using 3D data from sources such as Objaverse and fine-tuned with 2D style transfer tasks, endowing it with a rich 3D prior. This enables Snapmoji to transform a selfie into a primary stylized avatar, like the Bitmoji style, and apply a secondary style, such as Plastic Toy or Alien, all while preserving the user's identity and the primary style's integrity. Our system is capable of producing 3D Gaussian avatars that support dynamic animation, including accurate facial expression transfer. Designed for efficiency, Snapmoji achieves selfie-to-avatar conversion in just 0.9 seconds and supports real-time interactions on mobile devices at 30 to 40 frames per second. Extensive testing confirms that Snapmoji outperforms existing methods in versatility and speed, making it a convenient tool for automatic avatar creation in various styles.

Problem

• Instant generation of animatable, dual-stylized avatars from selfies.
• Overcoming limitations of existing avatar platforms in expressivity and customization.
• Efficient rendering and real-time interaction on mobile devices.

Innovation

• Instant generation of dual-stylized avatars
• Gaussian Domain Adaptation for 3D avatar creation
• Real-time animation at 30-40 fps on mobile