🤖 AI Summary
This study addresses the performance limitations of standard Transformers in point cloud dense prediction caused by weak prior assumptions. To overcome this, we propose PCT-Prompt, a novel framework incorporating a prompt-guided feature branch and a prompt dropout mechanism. By integrating a geometry-sensitive abstraction layer with cross-attention optimization, the method effectively fuses multi-scale geometric features with global regularization, achieving a dynamic balance between local details and global consistency. Extensive experiments on ShapeNetPart, S3DIS, and DALES datasets demonstrate that PCT-Prompt significantly improves dense prediction performance. These results validate the framework's effectiveness and robustness in handling complex 3D scenes, offering a promising solution for enhancing Transformer-based architectures in point cloud analysis tasks where strong inductive biases are typically lacking.
📝 Abstract
Standard Transformers have proven effective in point cloud object classification, but their performance in dense prediction tasks within complex scenes is often hindered by weak prior assumptions. To address this challenge, we propose PCT-Prompt, a novel framework that enhances standard Transformers by introducing a prompt-guided feature branch to improve performance in dense prediction tasks. The standard Transformer branch leverages pre-trained models for global feature extraction from point cloud data, serving as the backbone for processing high-level features. Meanwhile, the prompt-guided feature branch consists of two key components: a fine-grained feature extraction block that captures multi-scale geometric features using geometry-sensitive abstraction layer, along with the PnP-3D layer to integrate local context with global regularization. The second component, the prompt-refined feature learning block generates prompt tokens, which are subsequently refined through cross-attention mechanisms. Additionally, we introduce a prompt drop mechanism that progressively removes prompt information across Transformer layers, balancing local details and global consistency. Experimental results on the ShapeNetPart, S3DIS, and DALES datasets demonstrate that PCT-Prompt significantly improves the adaptability of standard Transformers to dense prediction tasks, achieving strong performance in real-world scenarios.