EvalMVX: A Unified Benchmarking for Neural 3D Reconstruction under Diverse Multiview Setups

2026-02-27

AI Summary

This work addresses the lack of unified benchmarks for high-fidelity 3D reconstruction methods beyond conventional multi-view stereo (MVS), particularly multi-view photometric stereo (MVPS) and multi-view shape-from-polarization (MVSfP). To this end, we introduce EvalMVX, a real-world dataset comprising 25 objects, 8,500 images captured from 20 viewpoints under 17 lighting conditions, and precisely aligned ground-truth 3D meshes. Using a polarization camera, we collect multi-modal data under both OLAT and natural illumination, enabling the first comprehensive benchmark that jointly evaluates MVS, MVPS, and MVSfP within a common framework. We systematically assess 13 state-of-the-art methods, revealing their performance limits across complex geometries and diverse materials. This dataset fills a critical gap in quantitative evaluation for multi-modal multi-view 3D reconstruction and establishes a reliable foundation for future research.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multi-instance/Multi-view Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Recent advancements in neural surface reconstruction have significantly enhanced 3D reconstruction. However, current real world datasets mainly focus on benchmarking multiview stereo (MVS) based on RGB inputs. Multiview photometric stereo (MVPS) and multiview shape from polarization (MVSfP), though indispensable on high-fidelity surface reconstruction and sparse inputs, have not been quantitatively assessed together with MVS. To determine the working range of different MVX (MVS, MVSfP, and MVPS) techniques, we propose EvalMVX, a real-world dataset containing $25$ objects, each captured with a polarized camera under $20$ varying views and $17$ light conditions including OLAT and natural illumination, leading to $8,500$ images. Each object includes aligned ground-truth 3D mesh, facilitating quantitative benchmarking of MVX methods simultaneously. Based on our EvalMVX, we evaluate $13$ MVX methods published in recent years, record the best-performing methods, and identify open problems under diverse geometric details and reflectance types. We hope EvalMVX and the benchmarking results can inspire future research on multiview 3D reconstruction.

Problem

• neural 3D reconstruction
• multiview stereo
• photometric stereo
• shape from polarization
• benchmarking

Innovation

• neural 3D reconstruction
• multiview benchmarking
• polarization imaging
• photometric stereo
• ground-truth 3D mesh