Diffusion Sequence Models for Generative In-Context Meta-Learning of Robot Dynamics

2026-04-14

AI Summary

This work addresses the challenge of inaccurate dynamics prediction in robotic systems under distribution shifts and real-time constraints by formalizing it as a contextual meta-learning problem. It proposes the first application of inpainting diffusion and conditional diffusion models to this domain, using a Transformer as a deterministic baseline. The approach leverages large-scale stochastic simulation combined with a warm-start sampling strategy to significantly enhance out-of-distribution generalization while meeting stringent real-time control latency requirements. Experimental results demonstrate that the proposed diffusion models—particularly the inpainting variant—exhibit markedly superior robustness under distribution shift compared to existing baselines, achieving both high predictive accuracy and practical applicability.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Intelligent Robots: State Estimation
• Machine Learning: Adversarial Learning & Robustness

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Accurate modeling of robot dynamics is essential for model-based control, yet remains challenging under distributional shifts and real-time constraints. In this work, we formulate system identification as an in-context meta-learning problem and compare deterministic and generative sequence models for forward dynamics prediction. We take a Transformer-based meta-model, as a strong deterministic baseline, and introduce to this setting two complementary diffusion-based approaches: (i) inpainting diffusion (Diffuser), which learns the joint input-observation distribution, and (ii) conditioned diffusion models (CNN and Transformer), which generate future observations conditioned on control inputs. Through large-scale randomized simulations, we analyze performance across in-distribution and out-of-distribution regimes, as well as computational trade-offs relevant for control. We show that diffusion models significantly improve robustness under distribution shift, with inpainting diffusion achieving the best performance in our experiments. Finally, we demonstrate that warm-started sampling enables diffusion models to operate within real-time constraints, making them viable for control applications. These results highlight generative meta-models as a promising direction for robust system identification in robotics.

Problem

• robot dynamics
• distribution shift
• real-time constraints
• system identification
• in-context meta-learning

Innovation

• diffusion models
• in-context meta-learning
• robot dynamics
• inpainting diffusion
• real-time control