SeeGroup: Multi-Layer Depth Estimation of Transparent Surfaces via Self-Determined Grouping

2026-05-27

AI Summary

This work addresses the challenge of multi-layer depth estimation for transparent objects, which requires simultaneous reconstruction of both the transparent surfaces and the scene geometry behind them. Existing approaches are limited by predefined layer ordering or grouping strategies that fail to capture the inherent ambiguity in depth layering. To overcome this, the authors propose an adaptive multi-layer depth estimation framework that models per-pixel depth as an unordered point process along camera rays. By formulating a permutation-invariant likelihood-based loss function, the model autonomously assigns surfaces to depth layers without requiring manual specification of layer order. Integrated within an end-to-end deep learning architecture, the method achieves a significant improvement on the LayeredDepth benchmark, raising the four-way relative depth accuracy from 61.34% to 70.09%, outperforming current state-of-the-art approaches.

Abstract

Transparent objects are common in daily life, and it is important to understand their multilayer depth, including the transparent surface and the objects behind it. Existing methods for multilayer depth typically extend single-layer prediction. They define layers by the front-to-back ordering of 3D points and predict the layers sequentially. However, as layered geometry can admit multiple valid groupings of 3D points into layers, a predefined grouping strategy is inherently restrictive. In this work, we propose SeeGroup, a multi-layer depth estimation method that avoids imposing a predefined grouping and allows the model itself to adaptively assign surfaces to depth maps. We formulate per-pixel multi-layer depth as a point process, treating depth layers as unordered events along each camera ray. This induces a permutation-invariant likelihood over the observed depth layers, yielding a loss that naturally supports arbitrary layer groupings. Experiments demonstrate that our method significantly advances the state of the art of multi-layer depth estimation, improving quadruplet relative depth accuracy on LayeredDepth benchmark from 61.34% to 70.09%. Code is available at https://github.com/princeton-vl/SeeGroup.

Problem

• multi-layer depth estimation
• transparent surfaces
• depth grouping
• 3D point grouping
• permutation-invariant depth

Innovation

• multi-layer depth estimation
• transparent surfaces
• self-determined grouping
• point process
• permutation-invariant likelihood