xTED: Cross-Domain Adaptation via Diffusion-Based Trajectory Editing

2024-09-13

AI Summary

To address data scarcity in the target domain for cross-domain decision-making, this paper proposes a diffusion-based trajectory editing framework that directly aligns source- and target-domain dynamics at the data level, bypassing complex policy transfer. Methodologically, it introduces a novel trajectory-level conditional denoising mechanism for joint modeling of states, actions, and rewards, augmented by a cross-domain dynamics consistency constraint to ensure semantic fidelity and dynamical reliability. Unlike conventional paradigms relying on domain-specific discriminators or representation learning, our framework is highly generalizable and lightweight. Experiments across multiple simulated environments and real-world robotic tasks demonstrate substantial improvements in both single-domain and cross-domain policy learning performance. The edited trajectories exhibit high state realism and strong dynamical consistency.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Diffusion Models for Vision
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Vertical and domain-specific search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Reusing pre-collected data from different domains is an appealing solution for decision-making tasks, especially when data in the target domain are limited. Existing cross-domain policy transfer methods mostly aim at learning domain correspondences or corrections to facilitate policy learning, such as learning task/domain-specific discriminators, representations, or policies. This design philosophy often results in heavy model architectures or task/domain-specific modeling, lacking flexibility. This reality makes us wonder: can we directly bridge the domain gaps universally at the data level, instead of relying on complex downstream cross-domain policy transfer procedures? In this study, we propose the Cross-Domain Trajectory EDiting (xTED) framework that employs a specially designed diffusion model for cross-domain trajectory adaptation. Our proposed model architecture effectively captures the intricate dependencies among states, actions, and rewards, as well as the dynamics patterns within target data. Edited by adding noises and denoising with the pre-trained diffusion model, source domain trajectories can be transformed to align with target domain properties while preserving original semantic information. This process effectively corrects underlying domain gaps, enhancing state realism and dynamics reliability in source data, and allowing flexible integration with various single-domain and cross-domain downstream policy learning methods. Despite its simplicity, xTED demonstrates superior performance in extensive simulation and real-robot experiments.

Problem

• Transfer Learning
• Data Scarcity
• Decision Learning

Innovation

• xTED framework
• domain adaptation
• noise augmentation