An Evaluation Framework for Generating Multi-View Images of a Person in a Scene

2026-09-03

AI Summary

本文针对生成多视角人物图像时空间一致性差的问题，提出了一种新的评估指标HSRD，并通过实验验证了其有效性。

Abstract

Recent generative image-editing Diffusion Transformers (DiTs) demonstrate impressive semantic editing capabilities but still struggle with spatially consistent camera angle changes. A primary bottleneck in training foundation models to execute free-form, promptable camera angle changes is the lack of specialized training data. While multi-view datasets exist for generic 3D environments and objects, there remains an absence of paired, multi-view datasets featuring human subjects at fixed locations in natural scenes, including frontal and side-profile views. Capturing such multi-camera data in unconstrained environments is logistically challenging and unscalable. In this paper, we first experiment with multiple state-of-the-art image editing models to create this data synthetically, but find that the outputs are frequently prone to hallucinations involving how much the subject's head turns relative to the background, often producing inconsistent environments. To address this issue, we propose the Head Scene Rotation Difference (HSRD) metric to quantitatively evaluate camera movements around a person. The proposed metric operates by decoupling camera movement from localized head pose manipulation. As demonstrated by the extensive experimentation, HSRD provides the pipeline necessary to evaluate 3D spatial parallax for a person in a scene, paving the way to reliably construct high-quality multi-view synthetic datasets.

Problem

• Multi-View Images
• Camera Angle Changes
• Training Data
• Human Subjects
• Natural Scenes

Innovation

• Head Scene Rotation Difference
• multi-view images
• generative image-editing
• Diffusion Transformers
• spatial consistency