InstantSticker: Realistic Decal Blending via Disentangled Object Reconstruction

2025-04-09

AI Summary

This work addresses three key challenges in 3D reconstruction: texture fusion artifacts and blurring, and low efficiency in editing and rendering. To this end, we propose a high-fidelity, editable decal fusion method. Methodologically, we introduce an optimizable shadow factor to decouple diffuse reflectance from shadow components; integrate ARAP-based mesh parameterization, local UV mapping, and neural texture enhancement for high-frequency detail recovery; and leverage the Disney BRDF model to enable millisecond-level albedo replacement. Technical innovations include image-based lighting (IBL) illumination decoupling, neural texture mapping, and a custom RVW metric quantifying local geometric distortion. Experiments demonstrate that our approach achieves state-of-the-art performance across editing fidelity, interactive latency, and real-time rendering efficiency—outperforming existing methods in all evaluated dimensions.

Technology Category

• Computer Vision: 3D Computer Vision
• Search and Optimization: Mixed Discrete/Continuous Search
• Machine Learning: Learning with Manifolds

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present InstantSticker, a disentangled reconstruction pipeline based on Image-Based Lighting (IBL), which focuses on highly realistic decal blending, simulates stickers attached to the reconstructed surface, and allows for instant editing and real-time rendering. To achieve stereoscopic impression of the decal, we introduce shadow factor into IBL, which can be adaptively optimized during training. This allows the shadow brightness of surfaces to be accurately decomposed rather than baked into the diffuse color, ensuring that the edited texture exhibits authentic shading. To address the issues of warping and blurriness in previous methods, we apply As-Rigid-As-Possible (ARAP) parameterization to pre-unfold a specified area of the mesh and use the local UV mapping combined with a neural texture map to enhance the ability to express high-frequency details in that area. For instant editing, we utilize the Disney BRDF model, explicitly defining material colors with 3-channel diffuse albedo. This enables instant replacement of albedo RGB values during the editing process, avoiding the prolonged optimization required in previous approaches. In our experiment, we introduce the Ratio Variance Warping (RVW) metric to evaluate the local geometric warping of the decal area. Extensive experimental results demonstrate that our method surpasses previous decal blending methods in terms of editing quality, editing speed and rendering speed, achieving the state-of-the-art.

Problem

• Achieving realistic decal blending with authentic shading
• Reducing warping and blurriness in decal application
• Enabling instant editing and real-time rendering of decals

Innovation

• Disentangled reconstruction pipeline with Image-Based Lighting
• ARAP parameterization and neural texture for high-frequency details
• Disney BRDF model for instant albedo replacement