I2PRef: Image-Driven Point Completion with Iterative Refinement

2026-05-26

AI Summary

This work addresses the challenge of high-quality 3D point cloud completion from a single RGB image without any 3D input. To this end, the authors propose a novel image-centric approach that first employs an Image-to-Point module to directly generate a complete point cloud from the input image, thereby explicitly establishing strong geometric priors. Subsequently, a Transformer-based Point-to-Point iterative refinement module leverages self-attention and cross-attention mechanisms to enable multiple rounds of interaction between the generated point cloud and image features for progressive enhancement. Notably, this method treats the image as the primary source of geometric information rather than a supplementary cue, eschewing conventional multimodal fusion strategies or auxiliary losses. On the ShapeNet-ViPC benchmark, the proposed approach achieves state-of-the-art performance, improving the Chamfer Distance metric by 12.3% over the current best method.

Abstract

We present an image-conditioned point cloud completion approach that treats images as the primary geometric source rather than a secondary guide. To this end, we introduce an Image-to-Point (I2P) module that can reconstruct complete point clouds directly from a single RGB image, with no need for 3D inputs. Additionally, we introduce a transformer-based Point-to-Point (P2P) refinement module that uses self- and cross-attention between point tokens and image features to iteratively refine the coarse I2P output. The I2P module enables the image encoder to learn rich geometric representations, while the P2P module progressively recovers fine-grained details. Unlike existing multimodal methods that rely on auxiliary losses or fusion modules, our explicit I2P task provides a strong, geometry-aware prior based on images alone. Extensive experiments on ShapeNet-ViPC demonstrate state-of-the-art completion performance with a 12.3% relative Chamfer Distance improvement over prior methods. Code is available at: https://github.com/AzharSindhi/I2PRef.git

Problem

• point cloud completion
• image-conditioned reconstruction
• 3D geometry
• single-view reconstruction
• Chamfer Distance

Innovation

• Image-to-Point
• Point Cloud Completion
• Iterative Refinement
• Transformer-based Attention
• Geometry-aware Prior