SLAT-Phys: Fast Material Property Field Prediction from Structured 3D Latents

2026-03-25

AI Summary

This work proposes a novel method for efficiently predicting spatially varying 3D material property fields—such as Young’s modulus, density, and Poisson’s ratio—from a single RGB image. By leveraging a pre-trained 3D generative model to extract structured 3D latent features, the approach employs a lightweight neural decoder to perform end-to-end regression of continuous material properties, entirely bypassing costly preprocessing steps like explicit 3D reconstruction or voxelization. As the first study to utilize structured 3D latent representations for material property prediction, the method achieves a significant computational advantage, requiring only 9.9 seconds per object on an NVIDIA RTX A5000—120 times faster than existing approaches—while maintaining competitive prediction accuracy.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Deep Generative Models & Autoencoders
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Estimating the material property field of 3D assets is critical for physics-based simulation, robotics, and digital twin generation. Existing vision-based approaches are either too expensive and slow or rely on 3D information. We present SLAT-Phys, an end-to-end method that predicts spatially varying material property fields of 3D assets directly from a single RGB image without explicit 3D reconstruction. Our approach leverages spatially organised latent features from a pretrained 3D asset generation model that encodes rich geometry and semantic prior, and trains a lightweight neural decoder to estimate Young's modulus, density, and Poisson's ratio. The coarse volumetric layout and semantic cues of the latent representation about object geometry and appearance enable accurate material estimation. Our experiments demonstrate that our method provides competitive accuracy in predicting continuous material parameters when compared against prior approaches, while significantly reducing computation time. In particular, SLAT-Phys requires only 9.9 seconds per object on an NVIDIA RTXA5000 GPU and avoids reconstruction and voxelization preprocessing. This results in 120x speedup compared to prior methods and enables faster material property estimation from a single image.

Problem

• material property estimation
• 3D assets
• physics-based simulation
• single RGB image
• spatially varying properties

Innovation

• material property estimation
• single-image 3D inference
• structured latent representation
• physics-based simulation
• fast neural decoding