Occlusion-robust Stylization for Drawing-based 3D Animation

2025-08-01

AI Summary

To address contour flickering, stroke blurring, and style degradation caused by limb occlusion in hand-drawn 3D animation, this paper proposes the Occlusion-Robust Stylization Framework (OSF). OSF is the first method to identify and bridge the “pose–style gap” inherent in stylization networks. It introduces a flow-guided edge generation mechanism and jointly optimizes optical flow estimation and stylization within a single-stage, end-to-end pipeline, enabling stable stylistic preservation under dynamic occlusions. Compared to state-of-the-art approaches, OSF achieves a 2.4× speedup in inference and a 2.1× improvement in memory efficiency. In complex occlusion scenarios, it significantly suppresses visual discontinuities—ensuring consistent rendering of coarse contours and distinctive stroke patterns while preserving artistic fidelity.

Technology Category

• Computer Vision: 3D Computer Vision
• Natural Language Processing: Sentiment Analysis, Stylistic Analysis, and Argument Mining
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• Economics, Online Markets and Human Computation: Fairness and ethical considerations in crowd work and in human-in-the-loop AI systems

Abstract

3D animation aims to generate a 3D animated video from an input image and a target 3D motion sequence. Recent advances in image-to-3D models enable the creation of animations directly from user-hand drawings. Distinguished from conventional 3D animation, drawing-based 3D animation is crucial to preserve artist's unique style properties, such as rough contours and distinct stroke patterns. However, recent methods still exhibit quality deterioration in style properties, especially under occlusions caused by overlapping body parts, leading to contour flickering and stroke blurring. This occurs due to a `stylization pose gap' between training and inference in stylization networks designed to preserve drawing styles in drawing-based 3D animation systems. The stylization pose gap denotes that input target poses used to train the stylization network are always in occlusion-free poses, while target poses encountered in an inference include diverse occlusions under dynamic motions. To this end, we propose Occlusion-robust Stylization Framework (OSF) for drawing-based 3D animation. We found that while employing object's edge can be effective input prior for guiding stylization, it becomes notably inaccurate when occlusions occur at inference. Thus, our proposed OSF provides occlusion-robust edge guidance for stylization network using optical flow, ensuring a consistent stylization even under occlusions. Furthermore, OSF operates in a single run instead of the previous two-stage method, achieving 2.4x faster inference and 2.1x less memory.

Problem

• Preserve artist's unique style in drawing-based 3D animation
• Address quality deterioration under occlusions in stylization
• Bridge stylization pose gap between training and inference

Innovation

• Uses optical flow for occlusion-robust edge guidance
• Single-run framework for faster inference
• Reduces memory usage by 2.1x