Gaussian-JEPA: Joint-Embedding Predictive Learning for 3D Gaussian Splats

2026-08-16

AI Summary

This study addresses the poor generalization in 3D Gaussian self-supervised learning caused by attribute coupling and proposes the Gaussian-JEPA framework. By replacing pixel-level reconstruction with latent space prediction, this method leverages a joint embedding architecture and multi-scale target generation to decouple geometric and appearance supervision, enabling efficient decoder-free representation learning. Experiments demonstrate that the model yields more consistent representations under resampling and partial observation conditions. Furthermore, frozen features from Gaussian-JEPA significantly outperform those from traditional reconstruction-based pretraining across downstream tasks, including shape completion, part segmentation, and classification. These results confirm that the proposed approach effectively enhances the versatility and reusability of 3D Gaussian representations, establishing a robust foundation for diverse 3D vision applications without requiring task-specific fine-tuning of the backbone encoder.

Abstract

3D Gaussian Splatting (3DGS) represents 3D content with anisotropic primitives that jointly encode geometry and appearance. Fixed-budget encoders consume sampled observations of Gaussian assets, so the same object may be observed through different primitive realizations. Existing self-supervised methods mainly reconstruct masked Gaussian attributes, tying supervision to one sampled realization and requiring an input-space decoder. Latent prediction offers an alternative, but its application to Gaussian tokens requires targets that accommodate coupled attributes and heterogeneous spatial support. We introduce Gaussian-JEPA, which predicts representations of held-out Gaussian token blocks from visible context. An online encoder processes the context, while a shared exponential-moving-average encoder supplies stop-gradient features for multi-scale targets. Complementary target projections and feature-space grounding provide latent supervision without reconstructing Gaussian attributes. We evaluate the features under Gaussian resampling, partial observations, and renderable shape completion, together with transfer to part segmentation and object classification. Compared with matched reconstruction pretraining, Gaussian-JEPA is more consistent across resampled inputs, retains more instance information under partial observations, and provides stronger frozen features for Gaussian completion. These results support latent prediction as an effective objective for reusable 3D Gaussian representations. Code is on the project page (https://amazingren.github.io/Gaussian-JEPA/).

Problem

• 3D Gaussian Splatting
• Self-supervised Learning
• Latent Prediction
• Joint-Embedding Predictive Learning
• Representation Learning

Innovation

• Joint-Embedding Predictive Learning
• 3D Gaussian Splatting
• Latent Prediction
• Self-Supervised Learning
• Exponential Moving Average