Charge: A Comprehensive Novel View Synthesis Benchmark and Dataset to Bind Them All

2025-12-15

AI Summary

Existing novel view synthesis (NVS) benchmarks lack high-quality dynamic scenes and multimodal ground truth, hindering robust training and fair evaluation of 4D reconstruction and neural rendering models. To address this, we introduce AnimNVS—the first NVS benchmark targeting animation-film-grade quality—by incorporating high-fidelity animated films into benchmark construction for the first time. We generate dense, multimodal ground truth including RGB, depth, surface normals, optical flow, and instance-level object segmentation. Furthermore, we propose a three-tiered evaluation protocol covering dense, sparse, and monocular settings. Leveraging differentiable camera modeling and physically consistent lighting-aware rendering, we synthesize over 100,000 high-fidelity frames. AnimNVS significantly improves generalization, geometric consistency, and motion modeling capability of NVS methods on complex dynamic scenes, establishing a unified evaluation standard for 4D scene understanding.

Technology Category

• Computer Vision: Computational Photography, Image & Video Synthesis
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

This paper presents a new dataset for Novel View Synthesis, generated from a high-quality, animated film with stunning realism and intricate detail. Our dataset captures a variety of dynamic scenes, complete with detailed textures, lighting, and motion, making it ideal for training and evaluating cutting-edge 4D scene reconstruction and novel view generation models. In addition to high-fidelity RGB images, we provide multiple complementary modalities, including depth, surface normals, object segmentation and optical flow, enabling a deeper understanding of scene geometry and motion. The dataset is organised into three distinct benchmarking scenarios: a dense multi-view camera setup, a sparse camera arrangement, and monocular video sequences, enabling a wide range of experimentation and comparison across varying levels of data sparsity. With its combination of visual richness, high-quality annotations, and diverse experimental setups, this dataset offers a unique resource for pushing the boundaries of view synthesis and 3D vision.

Problem

• Provides a high-quality dataset for novel view synthesis
• Enables training and evaluation of 4D scene reconstruction models
• Supports diverse benchmarking scenarios with multiple modalities

Innovation

• High-fidelity animated film dataset for novel view synthesis
• Includes depth, normals, segmentation, and optical flow modalities
• Three benchmarking scenarios: dense, sparse, and monocular setups