Search papers, labs, and topics across Lattice.
This paper introduces SierpinskiCam, a novel method for video retaking that enhances camera-controlled rendering by integrating Sierpinski triangle texture cues with traditional geometry-guided approaches. The method effectively addresses the challenges of sparse or missing regions when the target camera deviates from the source trajectory, achieving improved camera controllability and video quality. Experimental results demonstrate significant advancements in geometric consistency and visual fidelity across a variety of retaking scenarios, showcasing the potential of texture cues in video synthesis tasks.
SierpinskiCam transforms video retaking by leveraging rich texture cues to maintain quality and detail even when the camera angle shifts dramatically.
Generating novel renderings of a scene along user-defined camera trajectories from a single monocular video, dubbed video retaking, is a compelling but difficult problem in content creation and visual effects. Existing geometry-guided approaches reconstruct a 4D representation from the source video and render it along the target trajectory to condition video diffusion models. However, this guidance degrades as the target camera departs from the source trajectory, leaving newly revealed regions sparse or entirely missing. We propose SierpinskiCam, which addresses this limitation by augmenting geometry-based guidance with Sierpinski dome texture cues that contains rich trackable features even under large viewpoint changes. We further introduce a reference video conditioning mechanism that appends source-video tokens to the target-token sequence and separates the two streams with negative RoPE indices, enabling appearance grounding without architectural modification or per-video adaptation. Extensive experiments show that SierpinskiCam achieves significant gains in camera controllability, geometric consistency, and video quality across diverse and challenging retaking scenarios. Project page: https://hyelinnam.github.io/SierpinskiCam/.
