SplatPainter: Interactive Authoring of 3D Gaussians from 2D Edits via Test-Time Training

2025-12-04

AI Summary

Addressing the challenge of interactive editing for 3D Gaussian splatting assets—where existing diffusion- or optimization-based methods suffer from slow inference, degradation of original geometry/appearance consistency, and limited local controllability—this paper proposes a state-aware feedforward-finetuning co-design framework. Taking user-provided 2D views as input, it jointly optimizes compact Gaussian feature representations via test-time training to enable non-destructive local detail enhancement, mask-guided overpainting, and globally consistent recoloring. Without retraining or predefined editing templates, the method initiates lightweight iterative updates from a single feedforward prediction, achieving high-fidelity, multi-task-unified 3D Gaussian attribute editing at interactive frame rates. Crucially, it preserves the original asset’s identity and structural integrity throughout editing. (149 words)

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Feature Construction/Reformulation
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

The rise of 3D Gaussian Splatting has revolutionized photorealistic 3D asset creation, yet a critical gap remains for their interactive refinement and editing. Existing approaches based on diffusion or optimization are ill-suited for this task, as they are often prohibitively slow, destructive to the original asset's identity, or lack the precision for fine-grained control. To address this, we introduce ourmethod, a state-aware feedforward model that enables continuous editing of 3D Gaussian assets from user-provided 2D view(s). Our method directly predicts updates to the attributes of a compact, feature-rich Gaussian representation and leverages Test-Time Training to create a state-aware, iterative workflow. The versatility of our approach allows a single architecture to perform diverse tasks, including high-fidelity local detail refinement, local paint-over, and consistent global recoloring, all at interactive speeds, paving the way for fluid and intuitive 3D content authoring.

Problem

• Enabling interactive editing of 3D Gaussian Splatting assets from 2D views
• Overcoming slow, destructive, or imprecise existing 3D editing methods
• Achieving local refinement, painting, and global recoloring at interactive speeds

Innovation

• Test-Time Training for interactive 3D Gaussian editing
• State-aware feedforward model predicts Gaussian attribute updates
• Single architecture enables diverse tasks at interactive speeds