Color3D: Controllable and Consistent 3D Colorization with Personalized Colorizer

2025-10-11

AI Summary

Existing 3D coloring methods are restricted to static scenes and rely on averaging strategies to enforce multi-view consistency, resulting in impoverished color appearance and limited controllability. This paper introduces the first controllable 3D colorization framework applicable to both static and dynamic scenes, decomposing complex 3D coloring into a single-image colorization task. By learning scene-specific personalized colorizers supervised solely by Lab-color annotations from a single key view, our method enables differentiable and consistent color propagation across novel views and temporal frames within a Gaussian splatting representation. The framework supports seamless integration of arbitrary pre-trained image colorization models and preserves color diversity and user interactivity via lightweight fine-tuning. Experiments demonstrate significant improvements in cross-view and temporal consistency, as well as color richness, over state-of-the-art methods on multiple static and dynamic benchmarks.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

In this work, we present Color3D, a highly adaptable framework for colorizing both static and dynamic 3D scenes from monochromatic inputs, delivering visually diverse and chromatically vibrant reconstructions with flexible user-guided control. In contrast to existing methods that focus solely on static scenarios and enforce multi-view consistency by averaging color variations which inevitably sacrifice both chromatic richness and controllability, our approach is able to preserve color diversity and steerability while ensuring cross-view and cross-time consistency. In particular, the core insight of our method is to colorize only a single key view and then fine-tune a personalized colorizer to propagate its color to novel views and time steps. Through personalization, the colorizer learns a scene-specific deterministic color mapping underlying the reference view, enabling it to consistently project corresponding colors to the content in novel views and video frames via its inherent inductive bias. Once trained, the personalized colorizer can be applied to infer consistent chrominance for all other images, enabling direct reconstruction of colorful 3D scenes with a dedicated Lab color space Gaussian splatting representation. The proposed framework ingeniously recasts complicated 3D colorization as a more tractable single image paradigm, allowing seamless integration of arbitrary image colorization models with enhanced flexibility and controllability. Extensive experiments across diverse static and dynamic 3D colorization benchmarks substantiate that our method can deliver more consistent and chromatically rich renderings with precise user control. Project Page https://yecongwan.github.io/Color3D/.

Problem

• Colorizing monochromatic 3D scenes with user-guided control
• Ensuring color consistency across views and time steps
• Preserving chromatic richness while maintaining controllability

Innovation

• Colorizes single key view then propagates colors
• Fine-tunes personalized colorizer for scene-specific mapping
• Uses Gaussian splatting in Lab space for 3D reconstruction