Viewpoint Recommendation for Point Cloud Labeling Through Interaction Cost Modeling

2024-03-22

IEEE Transactions on Visualization and Computer Graphics

AI Summary

This work addresses the inefficiency of manual 3D point cloud semantic segmentation annotation, which suffers from frequent viewpoint adjustments and tedious lasso-based point selection. To mitigate this, the study introduces Fitts’ Law into the point cloud annotation process for the first time, proposing a viewpoint recommendation model grounded in interaction time cost. By optimizing viewpoint selection to minimize the time required for lasso operations, the method integrates a viewpoint optimization algorithm and an interaction cost modeling module into a 3D annotation system. Experimental results demonstrate that the proposed approach significantly reduces annotation time and outperforms existing viewpoint selection strategies across multiple datasets.

Technology Category

• Computer Vision: Segmentation
• Search and Optimization: Distributed Search
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Semantics and Knowledge: Representation, semantic annotation, enhancement, enrichments, access and/or integration of a variety of data on the Web
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience

Abstract

Semantic segmentation of 3D point clouds is important for many applications, such as autonomous driving. To train semantic segmentation models, labeled point cloud segmentation datasets are essential. Meanwhile, point cloud labeling is time-consuming for annotators, which typically involves tuning the camera viewpoint and selecting points by lasso. To reduce the time cost of point cloud labeling, we propose a viewpoint recommendation approach to reduce annotators’ labeling time costs. We adapt Fitts’ law to model the time cost of lasso selection in point clouds. Using the modeled time cost, the viewpoint that minimizes the lasso selection time cost is recommended to the annotator. We build a data labeling system for semantic segmentation of 3D point clouds that integrates our viewpoint recommendation approach. The system enables users to navigate to recommended viewpoints for efficient annotation. Through an ablation study, we observed that our approach effectively reduced the data labeling time cost. We also qualitatively compare our approach with previous viewpoint selection approaches on different datasets.

Problem

• point cloud labeling
• semantic segmentation
• viewpoint recommendation
• annotation time cost
• interaction cost

Innovation

• viewpoint recommendation
• interaction cost modeling
• Fitts' law
• point cloud labeling
• semantic segmentation