WarpHammer: Densifying Scene Warps with 3D Object Priors for Extreme View Synthesis

2026-06-30

AI Summary

Existing projection-based novel view synthesis methods often fail under large viewpoint changes due to scene distortion, sparse sampling, missing occluded surfaces, and mirror artifacts. This work proposes a training-free approach that, for the first time, enables seamless integration of external auxiliary views with unknown poses without fine-tuning the base model. By leveraging 3D generative priors (e.g., SAM3D) and pretrained multi-view geometric models, the method jointly processes reference and auxiliary images to construct a unified point cloud, effectively enhancing distorted regions, completing foreground geometry, and masking invalid background content to recover both appearance and camera cues. Evaluated across five benchmarks, the approach significantly improves synthesis stability and geometric fidelity under extreme viewpoint shifts, establishing the first scene-level novel view synthesis framework capable of naturally fusing pose-agnostic external views.

Abstract

Projection-conditioned novel view synthesis (NVS) warps an explicit 3D reconstruction of the input view into the target camera and conditions a generator on the warped rendering. This works well for small viewpoint changes but degrades sharply under large orbital motion: the warp becomes sparse around the orbited object, where hidden surfaces dominate the new view and mirror-like artifacts emerge, causing the generator to lose both pixel content and the implicit camera cue carried by the warp. We introduce WarpHammer, a training-free framework that resolves this failure mode by augmenting the warped scene with an explicit 3D reconstruction of the object obtained from a native 3D generative prior (e.g., SAM3D). The reconstructed object adds missing foreground surfaces and occludes background points that should no longer be visible, restoring both appearance and camera cues without fine-tuning the base model. The same explicit object representation further unlocks a capability current NVS pipelines do not support: incorporating auxiliary views of the object from sources outside the target scene, for example, a casual snapshot of a car paired with a manufacturer studio shot of the same model. We process the reference and auxiliary images jointly with a pretrained multi-view geometry foundation model, which predicts a unified point cloud that we fuse into the 3D object reconstruction. This yields substantially more faithful geometry than single-image reconstruction, without requiring user-provided camera poses for the auxiliary views. On five benchmarks, WarpHammer produces stable novel views at viewpoint deviations where strong baselines collapse, and is the first scene-level NVS method that can naturally fuse auxiliary, pose-unknown object views from an external source.

Problem

• novel view synthesis
• extreme view synthesis
• scene warping
• 3D reconstruction
• viewpoint change

Innovation

• novel view synthesis
• 3D object priors
• scene warping
• pose-agnostic fusion
• training-free framework