Label-Efficient Grasp Joint Prediction with Point-JEPA

2025-09-13

AI Summary

This work investigates whether 3D self-supervised pretraining can enable label-efficient regression of grasping joint angles. We propose a lightweight multi-hypothesis prediction framework based on Point-JEPA: a Point-JEPA encoder pretrained on ShapeNet extracts point-cloud representations; a winner-takes-all mechanism coupled with top-logit selection is introduced to construct a low-parameter multi-hypothesis output head. This design significantly mitigates overfitting under extreme data scarcity—achieving up to a 26% reduction in RMSE on the DLR-Hand II dataset using only 1% labeled data, matching the performance of fully supervised baselines. Our core contribution is the first adaptation of the Joint-Embedding Predictive Architecture (JEPA) paradigm to grasping pose regression, empirically validating the effectiveness and generalizability of unsupervised representation transfer for low-label 3D manipulation learning.

Technology Category

• Machine Learning: Learning with Manifolds
• Intelligent Robots: Manipulation
• Computer Vision: Representation Learning for Vision

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

We investigate whether 3D self-supervised pretraining with a Joint-Embedding Predictive Architecture (Point-JEPA) enables label-efficient grasp joint-angle prediction. Using point clouds tokenized from meshes and a ShapeNet-pretrained Point-JEPA encoder, we train a lightweight multi-hypothesis head with winner-takes-all and evaluate by top-logit selection. On DLR-Hand II with object-level splits, Point-JEPA reduces RMSE by up to 26% in low-label regimes and reaches parity with full supervision. These results suggest JEPA-style pretraining is a practical approach for data-efficient grasp learning.

Problem

• Label-efficient grasp joint-angle prediction
• Self-supervised 3D pretraining with Point-JEPA
• Data-efficient grasp learning with reduced supervision

Innovation

• Self-supervised 3D pretraining with Point-JEPA
• Lightweight multi-hypothesis head training
• Winner-takes-all top-logit selection evaluation