Probabilistic Directed Distance Fields for Ray-Based Shape Representations

2024-04-13

arXiv.org

AI Summary

3D shape representation faces a fundamental trade-off between geometric fidelity and rendering efficiency: explicit representations (e.g., meshes, point clouds) enable fast rendering but suffer from low fidelity, whereas implicit representations (e.g., SDFs, NeRFs) achieve high fidelity at the cost of slow, non-differentiable rendering. To address this, we propose the Directional Distance Field (DDF), which maps oriented 3D points to visibility and depth, enabling single-pass differentiable ray casting. We further introduce the Probabilistic DDF (PDDF) to model field discontinuities and support efficient extraction of differential quantities such as surface normals. Theoretically, we derive the minimal field constraints required for view consistency. Our method establishes the first ray-oriented, differentiable implicit shape representation. It significantly outperforms state-of-the-art approaches in single-shape fitting, generative modeling, and single-image 3D reconstruction—using only lightweight networks, producing depth estimates in one forward pass, and computing normals via a single backward pass.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Representation Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web query analysis, representation and understanding
• Security and Privacy: Data transparency and provenance

Abstract

In modern computer vision, the optimal representation of 3D shape continues to be task-dependent. One fundamental operation applied to such representations is differentiable rendering, as it enables inverse graphics approaches in learning frameworks. Standard explicit shape representations (voxels, point clouds, or meshes) are often easily rendered, but can suffer from limited geometric fidelity, among other issues. On the other hand, implicit representations (occupancy, distance, or radiance fields) preserve greater fidelity, but suffer from complex or inefficient rendering processes, limiting scalability. In this work, we devise Directed Distance Fields (DDFs), a novel neural shape representation that builds upon classical distance fields. The fundamental operation in a DDF maps an oriented point (position and direction) to surface visibility and depth. This enables efficient differentiable rendering, obtaining depth with a single forward pass per pixel, as well as differential geometric quantity extraction (e.g., surface normals), with only additional backward passes. Using probabilistic DDFs (PDDFs), we show how to model inherent discontinuities in the underlying field. We then apply DDFs to several applications, including single-shape fitting, generative modelling, and single-image 3D reconstruction, showcasing strong performance with simple architectural components via the versatility of our representation. Finally, since the dimensionality of DDFs permits view-dependent geometric artifacts, we conduct a theoretical investigation of the constraints necessary for view consistency. We find a small set of field properties that are sufficient to guarantee a DDF is consistent, without knowing, for instance, which shape the field is expressing.

Problem

• Develops Directed Distance Fields for efficient 3D shape rendering
• Addresses fidelity and scalability issues in implicit shape representations
• Ensures view consistency in neural shape representations theoretically

Innovation

• Directed Distance Fields for efficient rendering
• Probabilistic DDFs model field discontinuities
• Single forward pass per pixel depth