surface normal estimation

Neural implicit methods struggle to reconstruct fine-grained geometry, sharp edges, and thin structures from sparse multi-view RGB inputs—particularly with only two views (front and back). Method: Moving beyond conventional zero-order geometric constraints (e.g., point-projection consistency), we introduce first-order differential constraints—specifically surface normals—as explicit supervision for neural implicit modeling. We estimate monocular depth using Depth Anything and derive approximate image-space surface normals, formulating a normal consistency loss that jointly optimizes the first-order differentiable properties of signed distance functions (SDFs) or NeRF-like implicit fields. Results: Evaluated on both synthetic and real-world datasets, our method achieves high-fidelity 3D surface reconstruction from merely two RGB images. It significantly outperforms state-of-the-art approaches in PSNR, Chamfer distance, and visual quality, demonstrating that normal supervision is critical for recovering fine-scale geometric details.

Estimating surface normals from noisy point clouds remains a longstanding challenge, as existing methods rely heavily on clean inputs and strong supervision, failing to simultaneously preserve geometric details and ensure robustness. This paper proposes a local gradient-aware implicit surface filtering framework: it guides point cloud projection onto the underlying surface via an implicit function and introduces a joint normal-distance operator to enforce geometric constraints. To mitigate gradient degradation, we design a local gradient consistency loss; to suppress over-smoothing, we develop a multi-scale directional aggregation mechanism. Crucially, our method performs end-to-end learning without explicit normal supervision, unifying denoising, normal estimation, and surface reconstruction. Extensive experiments demonstrate state-of-the-art performance across multiple benchmarks, significantly improving fine-detail recovery under complex noise patterns. The code and pre-trained models are publicly available.

Data-driven 3D vision approaches—particularly deep learning methods—lack explicit, interpretable, and transferable geometric modeling capabilities. Method: This paper systematically investigates Gaussian curvature, a differential-geometric invariant, for monocular and binocular depth reconstruction. We propose Gaussian curvature as a compact surface descriptor and an unsupervised metric for stereo matching evaluation. Through theoretical analysis and empirical validation on the Middlebury dataset, we demonstrate that mainstream depth estimation models implicitly capture Gaussian curvature characteristics; further, explicitly incorporating Gaussian curvature as a prior significantly improves reconstruction accuracy and enables cross-modal transfer and controlled geometric ablation studies. Contribution/Results: To our knowledge, this is the first work to establish Gaussian curvature as a foundational, interpretable, structurally grounded, and evaluable 3D geometric representation. Our findings introduce a new paradigm for geometry-guided visual reconstruction, bridging differential geometry with modern data-driven vision.

To address inaccurate surface normal estimation in autonomous driving caused by sparse, non-uniform, and noisy LiDAR point clouds, this paper proposes the Hybrid Geometric Transformer (HGT). HGT is the first Transformer-based architecture to jointly model visual semantics and 3D geometric priors. It introduces a cross-modal feature alignment mechanism and a multi-modal self-supervised strategy to enable efficient geometric knowledge transfer from simulation to real-world scenes (e.g., KITTI). By fusing LiDAR point clouds with camera images through joint encoding and domain-adaptive fine-tuning, HGT significantly enhances geometric perception robustness. On KITTI, it achieves state-of-the-art accuracy in surface normal estimation—outperforming prior methods—and effectively supports downstream tasks such as collision avoidance and occlusion reasoning. Experimental results demonstrate strong generalization and practical applicability.

Reconstructing multiple objects from a single RGB-D frame faces core challenges including geometric unobservability in occluded regions, poor noise robustness, weak generalization, and miscalibrated confidence estimation. Method: We propose a retrieval-augmented Bayesian reconstruction framework. It introduces, for the first time, a cross-modal part-level retrieval mechanism that dynamically matches semantically similar parts from a pre-trained 3D mesh library to construct an information-rich, adaptive Bayesian prior—jointly modeling geometric uncertainty and occlusion reasoning. The framework integrates RGB-D perception, differentiable rendering, and probabilistic inference to yield well-calibrated reconstruction confidence. Contribution/Results: Evaluated on both synthetic and real-world cluttered scenes, our method significantly improves reconstruction accuracy in occluded regions and enhances robustness to sensor noise. Crucially, it enables dexterous robotic hands to achieve high-success-rate grasping, demonstrating practical applicability in complex, unstructured environments.

Sparse voxel-based reconstruction often suffers from surface fragmentation, excessive subdivision, and floating artifacts in regions with weak texture or insufficient observations. This work proposes a surface-adaptive reconstruction method that, for the first time, translates semantically and geometrically consistent 2D surface priors from image space into holistic 3D structural constraints throughout the entire pipeline. By integrating appearance cues, monocular depth, surface normals, and multi-view geometric information, the approach identifies structured regions and adaptively selects either planar or quadric surface models to guide voxel subdivision, optimization supervision, pruning, and post-processing. Evaluated on three public benchmarks, the method significantly outperforms existing techniques, achieving state-of-the-art and robust reconstruction quality—particularly under challenging visibility and geometric conditions.

This work addresses the challenge in multi-view normal fusion where varying camera distances cause inconsistent spatial scales of pixel coverage, leading to conflicting normals and loss of high-frequency geometric details. To resolve this, the authors propose a scale-encoded neural surface representation that explicitly models pixel coverage scale within a neural implicit framework for the first time. Each 3D point is associated with a local spatial scale, and scale-aware normals are computed via a hybrid grid encoding scheme. Furthermore, a scale-aware mesh extraction module is introduced to assign each vertex an optimal local scale based on observed data. The method significantly outperforms existing approaches under multi-distance capture conditions, effectively preserving normal consistency while achieving high-fidelity surface reconstruction with enhanced retention of fine-scale geometric details.

This work addresses the limitations of existing feedforward monocular 3D reconstruction methods in capturing fine-grained local surface geometry, which conventional evaluation metrics often fail to adequately quantify. To this end, we propose SurGe, a novel framework that introduces a point-wise surface normal-based metric to explicitly assess local geometric orientation errors. SurGe incorporates several key innovations, including a point gradient matching loss, a neighborhood-aware attention decoder, depth-normalized 3D finite differences, and a feature-progressive upsampling strategy. Extensive experiments demonstrate that SurGe achieves state-of-the-art performance on eight zero-shot monocular geometry benchmarks in terms of the global point-wise AbsRel metric, while consistently and significantly improving the fidelity of both local surface geometry and its corresponding normals.

This work addresses the challenge of unreliable disparity and surface normal estimation in complex scenarios—such as low-light conditions, highly reflective surfaces, and transparent objects—where existing feedforward stereo matching methods suffer from insufficient geometric priors. To overcome this limitation, we propose GeoStereo, a novel framework that introduces diffusion models to stereo geometric estimation for the first time. GeoStereo establishes a bidirectional guidance mechanism between disparity and surface normals through disparity-guided normal initialization and warping-based left-view alignment conditioning, enabling their joint optimization. Evaluated under unsupervised settings, our method significantly enhances geometric reconstruction robustness, achieving state-of-the-art zero-shot disparity estimation performance on KITTI and NYUv2, and delivering leading surface normal accuracy on real-world indoor benchmarks including iBims-1 and ScanNet.

Existing feedforward 3D reconstruction methods are largely confined to 2.5D representations of visible surfaces, struggling to efficiently recover complete geometric structures—particularly in occluded regions. This work proposes a unified feedforward framework based on sparse 3D queries, formulating 3D reconstruction for the first time as a sparse query inference problem. By introducing explicit geometric anchors as queries in global 3D space, the method leverages a decoupled cross-attention mechanism to enable multi-view feature interaction and employs differentiable rendering with 3D Gaussians. The approach reconstructs complete scenes—including occluded areas—in a single forward pass, significantly outperforming existing feedforward methods on the Mip-NeRF 360 and VR-NeRF datasets with orders-of-magnitude fewer primitives while achieving state-of-the-art results in both rendering quality and geometric accuracy.