Global Pose Control for Generative View Synthesis in Normalized Object Coordinate Space

2026-07-02

AI Summary

This work addresses the limitations of existing novel view synthesis methods, which often lack global and intuitive control over target viewpoints and typically rely on known input camera poses or support only sparse view generation. The authors reformulate novel view synthesis as an image editing task by generating views in the Normalized Object Coordinate Space (NOCS) conditioned on absolute camera poses, eliminating the need for input pose annotations. To enhance generalization, they introduce a text-guided NOCS alignment mechanism that defines a canonical object coordinate system via textual descriptions. Key contributions include the first method enabling global viewpoint control through absolute camera poses, the proposed text-guided NOCS alignment, and a high-quality NOCS dataset. The approach generates consistent, high-fidelity novel views from arbitrary pose-free images across diverse object categories, achieving state-of-the-art performance.

Abstract

Novel View Synthesis (NVS) enables the generation of unseen views of a scene from a single or multiple images, allowing users to freely explore an object from any viewpoint. Despite the recent impressive qualitative improvements of generative models for this task, existing methods struggle to provide global and intuitive control of target viewpoints because they either use input-relative camera poses or are limited to generating sparse global views. This lack of global pose control severely limits the number of downstream tasks potentially enabled by NVS. To address this limitation, we propose a novel approach for precise camera control in a customizable Normalized Object Coordinate Space (NOCS), requiring single or few unposed images. Our method operates solely on the absolute camera pose of the target view in NOCS, eliminating the need for a relative world frame or camera poses of the input images. Unlike previous methods that treat NVS as a standalone generation task, we formulate it as an image editing problem and build upon state-of-the-art editing models to leverage their superior generalization capability. Camera information is injected as dedicated camera tokens via an in-context multi-modal conditioning strategy. To alleviate the inherent ambiguity of NOCS, we incorporate text descriptions that explicitly define the object's canonical coordinate frame, which also enhances generalization to unseen object categories. Furthermore, we curate a high-quality dataset with consistently aligned orientations and corresponding NOCS text definitions. Extensive experiments demonstrate that our method robustly generates novel views with accurate and consistent orientations from arbitrary unposed images across diverse categories, achieving state-of-the-art image quality and fidelity.

Problem

• Novel View Synthesis
• Global Pose Control
• Normalized Object Coordinate Space
• Camera Pose
• Viewpoint Control

Innovation

• Normalized Object Coordinate Space
• Global Pose Control
• Novel View Synthesis
• In-Context Conditioning
• Image Editing