Point Cloud Sequence Encoding for Material-conditioned Graph Network Simulators

2026-05-20

AI Summary

Existing graph-based simulators rely on known material parameters and mesh reconstruction, limiting their deployment in real-world scenarios. This work proposes PEACH, a novel framework that, for the first time, integrates spatiotemporal point cloud sequence encoding with contextual learning to enable high-fidelity physical simulation without explicit material parameters or mesh reconstruction. PEACH adaptively infers unknown physical properties during inference by combining a graph network simulator with two auxiliary supervision mechanisms, facilitating efficient zero-shot simulation-to-reality (sim-to-real) transfer. Experimental results demonstrate that PEACH achieves superior prediction accuracy over mesh-based methods in dynamically complex scenes, significantly enhancing the practical feasibility of physics simulation in real-world applications.

Abstract

Graph Network Simulators (GNSs) have emerged as powerful surrogates for complex physics-based simulation, offering inherent differentiability and orders-of-magnitude speedups over traditional solvers. However, GNSs typically assume access to the underlying material parameters, such as stiffness or viscosity, severely limiting their utility in realistic experimental settings. While recent meta-learning approaches address the parameter dependency by inferring properties from mesh trajectories, reconstructing a mesh from an observed scene is challenging. In this work, we introduce Point Cloud Encoding for Accurate Context Handling (PEACH), a novel framework that applies in-context learning on point clouds to adapt a learned simulator to unseen physical properties during inference. Our approach relies on a novel spatio-temporal point cloud sequence encoder, as well as two forms of auxiliary supervision to help improve simulation fidelity. We demonstrate that PEACH is capable of accurate zero-shot sim-to-real transfer on a challenging, dynamic scene. Experiments on simulation scenes show that PEACH even outperforms mesh-based baselines on prediction accuracy, while being much more practical for real-world deployment.

Problem

• Graph Network Simulators
• material parameters
• point cloud sequence
• sim-to-real transfer
• in-context learning

Innovation

• point cloud sequence encoding
• graph network simulators
• in-context learning
• zero-shot sim-to-real transfer
• material-conditioned simulation