Variance Reduction on the Camera Axis: Multi-View Score Distillation for 3D

2026-06-29

AI Summary

Single-view score distillation in 3D generation suffers from high gradient variance and global shape inconsistency. This work proposes Multi-View Score Distillation Interpolation (MV-SDI), which aggregates gradients from antipodal view pairs to substantially reduce gradient estimation variance along the camera axis, while keeping the pre-trained 2D diffusion model frozen, requiring no multi-view data, and incurring no additional peak memory cost. Under a fixed UNet evaluation budget, MV-SDI with only K=2 views improves CLIP R-Precision to 83.8% and halves the number of optimization steps; with K=4 views, it reduces the step count by fourfold and achieves an R-Precision of 86.9%, significantly outperforming single-view baselines across all alignment metrics.

Abstract

Score distillation turns a pretrained 2D diffusion model into a 3D generator, but the per-step gradient is estimated from a single randomly chosen view: it is high-variance and blind to global shape consistency. Prior work addresses this by retraining the diffusion prior on multi-view data; this improves consistency but makes the sampling contribution inseparable from prior quality. We instead isolate the sampling axis. The per-step gradient is one noisy sample of an expectation over views; aggregating K samples per step at a fixed total UNet budget reduces variance without touching the prior. We introduce Multi-View Aggregated Score Distillation (MV-SDI), which aggregates gradients from K views per step via gradient accumulation, keeping peak memory unchanged and the 2D prior frozen, and draws views as antithetic antipodal pairs, a prior-independent geometric property, for balanced angular coverage. At a fixed 10,000-UNet-call budget, K=2 raises CLIP R-Precision from 74.8% to 83.8% and CLIP score from 0.297 to 0.312, with consistent gains on HPSv2 and ImageReward and a 0.0% divergence rate on the 43-prompt benchmark; optimization steps halve as a consequence. K=4 gives a fourfold step reduction at R-Precision 86.9% and CLIP 0.307, still well above the single-view baseline on every alignment metric. MV-SDI is compatible with gradient-based score-distillation pipelines, including Score Distillation via Inversion, and requires no retraining and no multi-view data.

Problem

• Score Distillation
• 3D Generation
• Variance Reduction
• Multi-View Consistency
• Gradient Estimation

Innovation

• score distillation
• variance reduction
• multi-view aggregation
• 3D generation
• gradient accumulation