DiffTrans: Differentiable Geometry-Materials Decomposition for Reconstructing Transparent Objects

2026-02-27

AI Summary

Transparent objects pose significant challenges in multi-view reconstruction due to their complex light transport properties, such as refraction and absorption. Existing methods often rely on oversimplified assumptions and struggle to handle intricate topologies and textures in real-world scenes. This work proposes DiffTrans, the first end-to-end differentiable rendering framework that jointly optimizes geometry, refractive index, and absorption coefficient. By integrating an initial reconstruction via FlexiCubes with recursive differentiable ray tracing, and introducing a dilate-smooth regularization scheme accelerated by CUDA, DiffTrans achieves substantial gains in both efficiency and accuracy. Experiments demonstrate that DiffTrans outperforms current state-of-the-art methods across multiple benchmarks, particularly excelling in high-fidelity reconstruction of complex transparent objects.

Technology Category

• Computer Vision: Interpretability, Explainability, and Transparency
• Machine Learning: Transparent, Interpretable, Explainable ML
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Data transparency and provenance
• Responsible Web: Algorithmic accountability and transparency on the web
• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization

Abstract

Reconstructing transparent objects from a set of multi-view images is a challenging task due to the complicated nature and indeterminate behavior of light propagation. Typical methods are primarily tailored to specific scenarios, such as objects following a uniform topology, exhibiting ideal transparency and surface specular reflections, or with only surface materials, which substantially constrains their practical applicability in real-world settings. In this work, we propose a differentiable rendering framework for transparent objects, dubbed DiffTrans, which allows for efficient decomposition and reconstruction of the geometry and materials of transparent objects, thereby reconstructing transparent objects accurately in intricate scenes with diverse topology and complex texture. Specifically, we first utilize FlexiCubes with dilation and smoothness regularization as the iso-surface representation to reconstruct an initial geometry efficiently from the multi-view object silhouette. Meanwhile, we employ the environment light radiance field to recover the environment of the scene. Then we devise a recursive differentiable ray tracer to further optimize the geometry, index of refraction and absorption rate simultaneously in a unified and end-to-end manner, leading to high-quality reconstruction of transparent objects in intricate scenes. A prominent advantage of the designed ray tracer is that it can be implemented in CUDA, enabling a significantly reduced computational cost. Extensive experiments on multiple benchmarks demonstrate the superior reconstruction performance of our DiffTrans compared with other methods, especially in intricate scenes involving transparent objects with diverse topology and complex texture. The code is available at https://github.com/lcp29/DiffTrans.

Problem

• transparent object reconstruction
• multi-view images
• light propagation
• geometry-materials decomposition
• complex scenes

Innovation

• differentiable rendering
• transparent object reconstruction
• geometry-material decomposition
• recursive ray tracing
• FlexiCubes