Taylor Series-Inspired Local Structure Fitting Network for Few-shot Point Cloud Semantic Segmentation

2025-04-03

AI Summary

Few-shot point cloud semantic segmentation confronts dual challenges: severe scarcity of annotated data and difficulty in modeling irregular local geometric structures; existing pretraining-based methods incur high computational overhead and neglect fine-grained geometric details. This paper proposes TaylorSeg—a pretraining-free framework that formulates local point cloud geometry as a Taylor series fitting problem. We introduce TaylorConv, the first convolutional operator that explicitly fuses low-order foundational and high-order refined geometric features via Taylor expansion. Furthermore, we design two variants: non-parametric TaylorSeg-NN for efficient nearest-neighbor matching, and parametric TaylorSeg-PN incorporating an adaptive Push-Pull module for robust cross-support-query feature alignment. Under the 2-way 1-shot setting, TaylorSeg achieves new state-of-the-art mIoU scores—improving by 2.28% on S3DIS and 4.37% on ScanNet. Notably, TaylorSeg-NN attains performance on par with leading parametric models despite requiring zero pretraining.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Learning to Search
• Machine Learning: Semi-Supervised Learning

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Few-shot point cloud semantic segmentation aims to accurately segment"unseen"new categories in point cloud scenes using limited labeled data. However, pretraining-based methods not only introduce excessive time overhead but also overlook the local structure representation among irregular point clouds. To address these issues, we propose a pretraining-free local structure fitting network for few-shot point cloud semantic segmentation, named TaylorSeg. Specifically, inspired by Taylor series, we treat the local structure representation of irregular point clouds as a polynomial fitting problem and propose a novel local structure fitting convolution, called TaylorConv. This convolution learns the low-order basic information and high-order refined information of point clouds from explicit encoding of local geometric structures. Then, using TaylorConv as the basic component, we construct two variants of TaylorSeg: a non-parametric TaylorSeg-NN and a parametric TaylorSeg-PN. The former can achieve performance comparable to existing parametric models without pretraining. For the latter, we equip it with an Adaptive Push-Pull (APP) module to mitigate the feature distribution differences between the query set and the support set. Extensive experiments validate the effectiveness of the proposed method. Notably, under the 2-way 1-shot setting, TaylorSeg-PN achieves improvements of +2.28% and +4.37% mIoU on the S3DIS and ScanNet datasets respectively, compared to the previous state-of-the-art methods. Our code is available at https://github.com/changshuowang/TaylorSeg.

Problem

• Few-shot segmentation of unseen point cloud categories with limited data
• Improving local structure representation in irregular point clouds
• Eliminating pretraining overhead for efficient semantic segmentation

Innovation

• TaylorConv for local structure fitting
• Pretraining-free TaylorSeg-NN and TaylorSeg-PN
• Adaptive Push-Pull module for feature alignment