Deep-JGAC: End-to-End Deep Joint Geometry and Attribute Compression for Dense Colored Point Clouds

2025-02-25

AI Summary

To address insufficient correlation exploitation and severe color-geometry mismatch in dense colored point cloud compression, this paper proposes an end-to-end joint compression framework. Methodologically, it introduces (1) attribute-assisted geometry encoding to explicitly model attribute-guided geometric structure; (2) an Attribute Information Fusion Module (AIFM) for cross-modal latent-space co-modeling; and (3) a geometry-distortion-adaptive recoloring module to mitigate color mismatch caused by geometric reconstruction errors. The framework supports plug-and-play integration of both learned and traditional encoders. Experiments demonstrate substantial improvements: average bit-rate reduction of 31.16%–82.96% over G-PCC and V-PCC under D1-PSNR; up to 57.14% gain in MS-GraphSIM; and significantly reduced encoding/decoding latency.

Technology Category

• Data Mining & Knowledge Management: Data Compression
• Computer Vision: Multi-modal Vision
• Machine Learning: Learning on the Edge & Model Compression

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Colored point cloud becomes a fundamental representation in the realm of 3D vision. Effective Point Cloud Compression (PCC) is urgently needed due to huge amount of data. In this paper, we propose an end-to-end Deep Joint Geometry and Attribute point cloud Compression (Deep-JGAC) framework for dense colored point clouds, which exploits the correlation between the geometry and attribute for high compression efficiency. Firstly, we propose a flexible Deep-JGAC framework, where the geometry and attribute sub-encoders are compatible to either learning or non-learning based geometry and attribute encoders. Secondly, we propose an attribute-assisted deep geometry encoder that enhances the geometry latent representation with the help of attribute, where the geometry decoding remains unchanged. Moreover, Attribute Information Fusion Module (AIFM) is proposed to fuse attribute information in geometry coding. Thirdly, to solve the mismatch between the point cloud geometry and attribute caused by the geometry compression distortion, we present an optimized re-colorization module to attach the attribute to the geometrically distorted point cloud for attribute coding. It enhances the colorization and lowers the computational complexity. Extensive experimental results demonstrate that in terms of the geometry quality metric D1-PSNR, the proposed Deep-JGAC achieves an average of 82.96%, 36.46%, 41.72%, and 31.16% bit-rate reductions as compared to the state-of-the-art G-PCC, V-PCC, GRASP, and PCGCv2, respectively. In terms of perceptual joint quality metric MS-GraphSIM, the proposed Deep-JGAC achieves an average of 48.72%, 14.67%, and 57.14% bit-rate reductions compared to the G-PCC, V-PCC, and IT-DL-PCC, respectively. The encoding/decoding time costs are also reduced by 94.29%/24.70%, and 96.75%/91.02% on average as compared with the V-PCC and IT-DL-PCC.

Problem

• Compress dense colored point clouds efficiently
• Enhance geometry with attribute information
• Optimize re-colorization for distorted geometry

Innovation

• End-to-end deep joint compression
• Attribute-assisted geometry encoder
• Optimized re-colorization module