🤖 AI Summary
Traditional grid-based domain randomization struggles to effectively handle the intricate textures and geometries of complex organisms such as insects, limiting sim-to-real transfer performance. This work proposes the first mesh-free domain randomization framework, which performs explicit perturbations directly in the parameter space of 3D Gaussian splatting. Photometric variations are achieved by modulating spherical harmonics coefficients, original appearances are replaced with 3D procedural noise, and randomized background synthesis is integrated. By eliminating reliance on mesh models, the method provides an efficient data augmentation strategy for complex geometries, substantially narrowing the gap between simulated and real domains and yielding training data with enhanced robustness and superior generalization capabilities.
📝 Abstract
Domain Randomization (DR) is a standard technique for closing the Sim-to-Real gap, yet traditional DR pipelines rely on classical computer graphics rendering driven by polygon meshes. For complex organic subjects, such as insect specimens, extracting and rendering textured meshes is challenging. To address this issue, we propose a meshless DR framework that operates on the parameter space of 3D Gaussian Splatting (3DGS). Our method employs two independent perturbation pipelines to synthesize randomized training datasets. First, a Photometric DR pipeline alters the baked illumination and color balance by modulating the Spherical Harmonics (SH) coefficients. Second, a Procedural DR pipeline isolates the subject's geometric shape by replacing its original textures with 3D spatial noise. Finally, these perturbed radiance fields are composited over stochastically varied backgrounds using a rasterization engine. Our parameter manipulation provides a meshless alternative for generating robust datasets for complex geometries.