DiffTetVR: Differentiable Tetrahedral Volume Rendering

2025-12-31

arXiv.org

AI Summary

This work addresses differentiable volume rendering on tetrahedral meshes to enable joint optimization of geometry and appearance from images. Moving beyond the limitations of regular voxel grids, the authors propose an end-to-end differentiable tetrahedral volume rendering framework that integrates efficient forward rendering, accurate gradient backpropagation, regularization for degenerate tetrahedra, and a local adaptive refinement mechanism. The method supports direct optimization of vertex positions and facilitates coarse-to-fine mesh reconstruction, achieving high-quality joint recovery of geometry and appearance even in complex scenes. The open-sourced implementation demonstrates the approach’s effectiveness and practical utility.

Technology Category

• Computer Vision: 3D Computer Vision
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning
• Machine Learning: Learning with Manifolds

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Security and Privacy: Data transparency and provenance

Abstract

Differentiable rendering is a technique that aims to invert the rendering process to enable optimizing rendering parameters from a set of images. In this article, we present a differentiable volume rendering solution called DiffTetVR for tetrahedral meshes. Unlike previous works based on regular grids, this enables the optimization of vertex positions and the local subdivision of the mesh without relying on multigrid methods. We present an efficient implementation of the forward rendering process, deduce the derivatives for the backwards pass and regularization terms for avoiding degenerate tetrahedra, and finally show how the tetrahedral mesh can be subdivided locally to enable a coarse-to-fine optimization process. The source code is made publicly available on GitHub at https://github.com/chrismile/DiffTetVR.

Problem

• differentiable rendering
• tetrahedral mesh
• volume rendering
• mesh optimization
• 3D reconstruction

Innovation

• differentiable rendering
• tetrahedral mesh
• volume rendering
• mesh optimization
• local subdivision