Object-centric Task Representation and Transfer using Diffused Orientation Fields

2025-11-23

AI Summary

Curved objects lack a global reference frame, hindering generalization of orientation-dependent tasks—e.g., “cutting along the surface”—across diverse shapes. To address this, we propose Diffusion-based Orientation Fields (DOF): a continuous, local reference frame representation generated online from raw point clouds via a diffusion-driven partial differential equation. DOF reformulates surface manipulation tasks as sparse keypoint-guided orientation field matching, eliminating reliance on global coordinates or topological consistency. Consequently, it enables skill transfer across geometrically heterogeneous, topologically variable, and spatially perturbed objects. Evaluated on contact-intensive tasks—including detection, cutting, and peeling—DOF significantly improves policy generalization and robustness on non-planar objects. Crucially, it achieves zero-shot cross-shape skill transfer without retraining.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: Manipulation
• Machine Learning: Learning with Manifolds

Application Category

• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Scalable techniques for the creation, curation, publication, maintenance, and consumption of large, Web-based, structured, reusable, knowledge graphs and ontologies

Abstract

Curved objects pose a fundamental challenge for skill transfer in robotics: unlike planar surfaces, they do not admit a global reference frame. As a result, task-relevant directions such as "toward" or "along" the surface vary with position and geometry, making object-centric tasks difficult to transfer across shapes. To address this, we introduce an approach using Diffused Orientation Fields (DOF), a smooth representation of local reference frames, for transfer learning of tasks across curved objects. By expressing manipulation tasks in these smoothly varying local frames, we reduce the problem of transferring tasks across curved objects to establishing sparse keypoint correspondences. DOF is computed online from raw point cloud data using diffusion processes governed by partial differential equations, conditioned on keypoints. We evaluate DOF under geometric, topological, and localization perturbations, and demonstrate successful transfer of tasks requiring continuous physical interaction such as inspection, slicing, and peeling across varied objects. We provide our open-source codes at our website https://github.com/idiap/diffused_fields_robotics

Problem

• Transferring robotic skills across curved objects lacking global reference frames
• Expressing manipulation tasks using smoothly varying local orientation fields
• Establishing sparse keypoint correspondences for continuous physical interactions

Innovation

• Using diffused orientation fields for local reference frames
• Transferring tasks via sparse keypoint correspondences between objects
• Computing orientation fields online from point clouds using diffusion