SierpinskiCam: Camera-Controlled Video Retaking with Sierpinski Triangle Pattern Cues

2026-06-15

AI Summary

This work addresses the challenge of novel view synthesis from monocular videos under user-specified camera trajectories, where target views deviating significantly from the source trajectory often lead to failed geometry guidance and sparse or missing newly exposed regions. To tackle this, the paper introduces—for the first time—the Sierpiński triangle dome fractal texture as an auxiliary guide for cross-view feature tracking. Combined with a reference-video conditioning mechanism and a dual-stream token fusion strategy employing negative RoPE indexing, the approach achieves appearance anchoring and high-quality re-rendering without modifying the model architecture or requiring per-video fine-tuning. The proposed method substantially enhances camera controllability, geometric consistency, and visual fidelity in complex re-photography scenarios.

Abstract

Generating novel renderings of a scene along user-defined camera trajectories from a single monocular video, dubbed video retaking, is a compelling but difficult problem in content creation and visual effects. Existing geometry-guided approaches reconstruct a 4D representation from the source video and render it along the target trajectory to condition video diffusion models. However, this guidance degrades as the target camera departs from the source trajectory, leaving newly revealed regions sparse or entirely missing. We propose SierpinskiCam, which addresses this limitation by augmenting geometry-based guidance with Sierpinski dome texture cues that contains rich trackable features even under large viewpoint changes. We further introduce a reference video conditioning mechanism that appends source-video tokens to the target-token sequence and separates the two streams with negative RoPE indices, enabling appearance grounding without architectural modification or per-video adaptation. Extensive experiments show that SierpinskiCam achieves significant gains in camera controllability, geometric consistency, and video quality across diverse and challenging retaking scenarios. Project page: https://hyelinnam.github.io/SierpinskiCam/.

Problem

• video retaking
• camera trajectory
• monocular video
• novel view synthesis
• geometric guidance

Innovation

• video retaking
• Sierpinski triangle
• camera trajectory
• geometry-guided rendering
• reference video conditioning